how to submit a form programmatically

I have received some comments about submitting a form from your applications, and I’ve decided to write an article about that.

There are a number of ways to accomplish this task:

• you can use to record a session — this will generate all the code for everything you do inside your browser: clicking a button, filling out a certain field. This is an easy solution if you don’t know programming
• you can use the firefox addon to find out all the names of the fields your browser is sending, and their values as well. A downside would be that you need to submit the form at least once.
• you can use the firefox addon to find the names of the fields in a form. A downside would be that it’s very likely to miss some of them because they are hidden. This is why I would recommend using tamper data together with firebug.

I will illustrate this process with screenshots and some code:

Here is what we will be submitting:

And this is the html for it:

 action="whatever.php" method="POST">
 

  for="user">User
  type="text" name="user"/>
 
 

  for="pwd">Pass
  type="password" name="pwd"/>
 
  type="submit" value="login"/>

( please, don’t even bother telling me that this html code doesn’t respect the standards. I don’t care. This is for learning purposes only )

It’s a simple form made of three fields : username, password and the submit button. Open your favorite text editor and paste it in. Save the buffer to a file ending with .html extension, then open it in your browser.

I hope you installed tamper data and firebug, because now we’ll make use of them. We’ll start with firebug. If you’ve installed it, a bug like icon will appear in the lower right corner of the browser. If it’s coloured gray, it means it’s disabled, and you have to click it and enable all of it’s features. If you’ve succeeded in doing that, the icon should be now orange, with black stripes.

Right click the user field. The contextual menu should have the option “Inspect Element”, like in the following screenshot:

Click it. You should now see something resembling this picture:

Notice that the field’s name is “user”. If you do the same for the password field, you’ll see that it’s name is “pass”. In this example, this is redundant, because we already know the name of the fields. However, in the real-world, you will not, and you should follow the steps showed here. Here is the code we have so far :

require "rubygems"
require "mechanize"

mech = WWW::Mechanize.new
# i'm loading this file locally
# in real-life you would provide the url of the page containing the form you want to submit
mech.get("file:///test_files/form_test.html")
# obtain the form object
# because this page contains only one form, it's obvious we request the first one
# if the page contained more than one form, you would have iterated over the forms
# and selected the one containing the fields you needed
form = mech.page.forms.first
# and now we complete the fields
# username first
# the order in which you complete this form is not important
form.user = "geo"
# and now the password
form.pwd = "mypassword"
# submit the form
form.submit
# do whatever you want to with the returned page
puts mech.page.body

If you run this code you’ll notice that it works ( that is, if you configured the action parameter to something real. If you haven’t, you’ll get a 40* error code, which still means that it works - this error will appear because the script needed to handle the form wasn’t found )

Usually, before submitting a form, you should use tamper data to make sure you’re sending all the parameters. So, open the website in firefox, fill out all the fields in the form, go to the “Tools” menu entry of your browser, click “Tamper Data”, like in the following screenshot :

If you did this, a new window will appear on your desktop :

Click “Start tamper”, and then submit your form ( click on login/submit/search/whatever ). After you’ve done this, something like this will appear :

Click Tamper. This is what you will see next :

In this example, this is exactly what we expected to see. Just the user and pwd fields are sent. However, in the real-world, you’ll see that usually more parameters are needed. Use tamper data before you start writing your code.

I like using mechanize for this sort of stuff, because it really makes this sort of tasks easy for you to handle. You can apply what you’ve learned here to whatever “mechanize-like framework”.

kthxbai