Using curl to sent an http/https request with a large cookie (more than 17 KB)

Using curl to sent an http/https request with a large cookie (more than 17 KB)

  

Using curl, we can send an http/https request with a large cookie (more than 17 KB). Now let’s see how to achieve this.

 

Example:

1.       Get and store cookies from Google.com

# curl -c "google_cookies.txt" -k -x 10.226.28.1:8080

-c, save Google cookies to file "google_cookies.txt"

-k, do not do Certification verification (Optional)

-x, use a proxy. (Optional)

 

File "google_cookies.txt" might look like this:

[WCG-7.1.6-1231 @~]#cat google_cookies.txt

# Netscape HTTP Cookie File

#

# This file was generated by libcurl! Edit at your own risk.

 

.google.com     TRUE    /       FALSE   1348541907      PREF    ID=6e26cbde6e66254f:NW=1:TM=1285469907:LM=1285469907:S=o-hTi7RHjslBPUFL

.google.com     TRUE    /       FALSE   1301281107      NID     39=D341jPM4INWKT_4mUdoAaaNN83IpfifQCN84DBolPVIjYUNBESXO5woRwgS3SrYUmUcVKRoVVLSSrP8ZypmA7iINYDVpyrA-dcvdcUxTOXxxCPOTGTjnbWmv86ILu6Us

 

NOTE:

1). You can edit this cookie file to set your cookies for a specific http/https site.

2). the delimiter between each field can NOT be spaces, it can be just a TAB, otherwise, curl will not tread that line as a cookie line when you use the '-b' option.

3). When modify this cookie file, make sure to use the domain which your URL rely on, otherwise, curl will not pick up this cookie line.

 

2.  Specify cookie to curl

When use '-b' option with a cookie file, curl will pickup those lines which with the same domain to the target URL.

# curl -b "226.28.5_cookies.txt" -k -x 10.230.21.107:8080

This command will pick up those cookie lines lead by 10.226.28.5.

 

[WCG-7.1.6-1231 @~]#cat 226.28.5_cookies.txt

# Netscape HTTP Cookie File

#

# This file was generated by libcurl! Edit at your own risk.

 

10.226.28.5     TRUE    /       FALSE   1348541907      PREF    ID=6e26cbde6e66254f:NW=1:TM=1285469907:LM=1285469907:S=o-hTi7RHjslBPUFL

.google.com     TRUE    /       FALSE   1301281107      NID     39=D341jPM4INWKT_4mUdoAaaNN83IpfifQCN84DBolPVIjYUNBESXO5woRwgS3SrYUmUcVKRoVVLSSrP8ZypmA7iINYDVpyrA-dcvdcUxTOXxxCPOTGTjnbWmv86ILu6Us

 

NOTE: WCG and curl can support 17+ KB cookies, but apache will not support this long cookie by default, this 2 commands in httpd.conf can modify that, but the MAX number you can set is a compile time configuration.

LimitRequestLine 18432

LimitRequestFieldSize 18432

 

We can also use curl to set a sort cookie:

# curl -b "NAME1=VALUE1; NAME2=VALUE2" -k -x 10.230.21.107:8080

 

3.  Addenda

More details of -b/-c option from man page of curl

-b/--cookie

              (HTTP)  Pass the data to the HTTP server as a cookie. It is supposedly the data previously received from the server in a "Set-Cookie:" line.

              The data should be in the format "NAME1=VALUE1; NAME2=VALUE2".

 

              If no ’=’ letter is used in the line, it is treated as a filename to use to read previously stored cookie lines from, which should  be  used

              in  this session if they match. Using this method also activates the "cookie parser" which will make curl record incoming cookies too, which

              may be handy if you’re using this in combination with the -L/--location option. The file format of the file to read cookies from  should  be

              plain HTTP headers or the Netscape/Mozilla cookie file format.

 

              NOTE  that  the  file  specified  with  -b/--cookie  is only used as input. No cookies will be stored in the file. To store cookies, use the

              -c/--cookie-jar option or you could even save the HTTP headers to a file using -D/--dump-header!

 

              If this option is set more than once, the last one will be the one that’s used.

 

-c/--cookie-jar

              Specify  to which file you want curl to write all cookies after a completed operation. Curl writes all cookies previously read from a speci-

              fied file as well as all cookies received from remote server(s). If no cookies are known, no file will be written. The file will be  written

              using the Netscape cookie file format. If you set the file name to a single dash, "-", the cookies will be written to stdout.

 

              NOTE  If  the  cookie jar can’t be created or written to, the whole curl operation won’t fail or even report an error clearly. Using -v will

              get a warning displayed, but that is the only visible feedback you get about this possibly lethal situation.

 

              If this option is used several times, the last specfied file name will be used.