Tor browser proxy for specific country.

Tor browser logo
Image from: http://4.bp.blogspot.com/-61TeilBY4fo/VRm1CF88mxI/AAAAAAAADYU/eGlnvX7eMRw/s1600/Tor.png

Recently, I had a task for testing some country specific features and I found out that there’s a limited amount of free services to set up a proxy for a predefined country. Mostly, people use proxies, because they don’t want to have their IPs from a specific country, due to country regulations and so on, but having an IP from specific country is a bit reverse logic, but as testers we have to do stuff like this, sometimes. So, one way to do so, of course having limitations on its own, is using Tor browser.

What’s Tor and Tor browser in first place.

I am not pretending to be extremely familiar with Tor and all it specifics, but on a pretty high level Tor is a network of proxy nodes. It has a couple of different node types – relay, bridge and the Tor browser itself. What it does, simply said, is to bounce your connection between many such “nodes” in the Tor network, to hide the origin of the connection e.g. make it anonymous. There’s a lot of talk and disputes, whether or not Tor really provides internet anonymity or it’s a dud, if it is reliable and so on, but for the purpose of this blog post, I won’t dive deeper into details about Tor. The reason why I am using it – it works as proxy and we can manipulate it in order to perform some localization testing.

Downloading Tor browser.

You can download Tor browser from its official page – Downloads. The installation is pretty straight forward, for Windows there’s a .exe file you can install and on Linux there’s a .gz archive you can download and place in whatever directory you need it. I will be mainly focused on Linux, since that’s what I used, but things shouldn’t differ in the other platforms as well.

Starting Tor browser.

That’s fairly simple as well, in the directory where you downloaded it, there.s a .desktop file which invokes a run script, so you can simply double-click it, or just run it as a shell script with “./”. Now, if you want to see what your address is you could navigate to: https://www.dnsleaktest.com/ and check your address. With each restart of the browser you will find, that a random IP address is assigned. But there’s a trick to use, to make our IP-based on a specific location.

Manipulating country settings.

For that purpose you will need to navigate to the following folder in your Tor browser installation folder:

~/tor-browser_en-US/Browser/TorBrowser/Data/Tor and what you need is the file called torcc. When you open it you will see something like that:

Let’s say we want to set our IP to UK. For that purpose we need to add the following two lines:

We can practically put any country code in the brackets in order to make it work, for a specific country. Save the file, start Tor browser again and check your IP again. It’s set to UK, voila!

Known issue:

Since the Tor network and Tor browser are not proprietary software and the Tor nodes are simply machines set up by hobbyists. We can’t rely any location will be available to set our proxy to. The network needs some minimum amount of nodes to the desired location in order to establish a full circuit. So, for some countries you might not be able to connect.

So, hope that article was interesting and informative for you, if you liked it don’t forget to comment and share. Thanks 😉

WordPress docker container on your local PC.

wordpress docker container
Image by: http://blog.loadimpact.com/wp-content/uploads/2014/12/WordPress-Docker.png

I have been thinking this for a long time, but I didn’t have the time to realize it, but now I am done with it and I wanted to share my progress. Since the Linux containers is the next huge thing in administering test environments I really wanted to make a copy my blog locally and I wanted to use a wordpress docker container for that purpose.

In order to do this we are going to need a running wordpress site – hosted somewhere, a docker installation and a WP plugin called Duplicator.

Here’s the small plan of actions to do it:

  • Installing docker.
  • Making a package of your existing WP installation via Duplicator.  
  • Move the package to the wordpress docker container.
  • Run the installer. 
  • Fix what you’ve just fucked up. 

What is docker container in first place.

I see that there’s lot of confusion on what a container is, so I will try to explain it in the way that I understand it. Think of a container as of an extremely trimmed and minimized version of a Linux OS, running only the utilities that you need. Practically, it comes to serve in the same manner as virtualization would do, but having a lot of advantages over it – no hypervisor, no boot times for the OS, the component you run in a container has all you need and nothing more. Docker offers all these pre-compiled docker images for almost everything you might need, including wordpress.

Installing Docker.

I am not going to explain this, since it’s more of a docker related info. I would recommend you to visit Docker’s official page where there’s a short tutorial on how to do that including a lot of documentation for using docker.

Note: I am using Linux (Lubuntu) as host for the Docker container.

Running a wordpress docker container.

I had some confusion with this, in fact I thought every time I want to have to container up, I have to run it. Which is incorrect so here’s a small walk through in container management.

You run the commands in exactly the same order and this should start a running wordpress docker container on 127.0.0.1:8282. You might want to ensure everything is running smoothly, just check the log, if mySql was installed w/o errors and Apache is running, it’s all OK, for now.

Once you have your container running you can bring it up/down and restart it with one of the following commands:

If you have any more questions, I used this article as guide for doing it.

Creating a package of your existing site.

For this purpose we’ll use the plugin called Duplicator. Duplicator is a plugin that let’s you to make a backup of your site with all its existing assets – plugins, database, media, themes, etc and deploy it on an empty machine via install.php script that the plugin creates. I repeat empty machine, because I am almost sure you don’t need wordpress installed on your new host, it will be created by the script. Anyway, I would risk deploying it on an empty Ubuntu server for example since I am not quite sure if it will install the MySql, Apache and so on. That’s why, we use already existing infrastructure of a wordpress and we’ll just deploy the new site there.

So, on the plugin. When installed it will appear in your menu and when opened you will be prompted to create your first package. The screens are pretty straight forward so just navigate through them and when done you will have two files that you will need – installer.php file and an archive.

duplicator

Now you need to download both these locally and put them into the wordpress docker container. Let’s say its name is “freddy”. using this installation, all your wordpress related files are stored in /app folder and that’s the place you need these two as well. You can easily do this by executing the following command.

Once you have these, you have these don’t be to quick on running the installer ’cause you have couple more thing to do. First, make sure the installer.php file has executable rights. To me, it wasn’t a problem to give it 777 since it’s a local machine and I will only use it for testing purposes, but in your case you might need to consider something else. The second thing, we need to do is to eighter delete the wp-config.php or the whole directory content, besides the installer and the archive. Other wise the installer will complain cause “there’s another wp installation there already and bla bla bla”. To me, deleting the wp-config file worked well enough. So, executing commands to a container might be done in two ways.

First, you can execute command once via the exec command like this:

Or you could sort of log in or attach to the container:

And you can directly navigate within your container and do the rest:

That should do. Now, you can go to 127.0.0.1:8282/installer.php and run the installer. In order to proceed you will need to kknow the instance of your database, pass and user. If you don’t know them you can simply cat the wp-config.php(before deleting it) and see in there.

Fixing what we’ve fucked up.

Normally when you finish the installation, there’s couple of thing you’ll have to do.

The plugin itself will ask you to update your symlinks, which is good as well as deleting the installer and the package, to avoid security breaches.

In my case, the site loaded all well, but some of my plugins we’re broken. For example – w3cache got completely messed up and all my styles were fucked, I had to remove it in order to have the site looking normally. Also, Crayon Syntax Highlighter got completely broken, so I removed it, too. So, these are kind of expected things that might get broken, after you move the site. Of course I don’t think they are something serious and are probably fixable by cleaning their cache, but for my purposes that was just useless.

So, I hope this post was helpful and interesting, if you liked it don’t forget to comment and share it with your friends. Good luck 🙂

Learning Linux as a tester.

Learning Linux- may the source be with you
Source of the image: www.geekstir.com

I’ve been a Linux user for a while and I’ve been a tester for a while and I always thought that the two are basically connected and since I am mostly testing in Linux environment right now I decided to state publicly my opinion on why learning Linux is vital for testers and how they could benefit from this.

Why is learning Linux is important for you as a tester?

We all switch positions occasionally and when we find a suitable offer in technical skills part it says something like that:

  • Good knowledge of network protocols (TCP/IP, ICMP, UDP, etc)
  • Good knowledge of at least one scripting language is a plus (Python, Perl, Ruby)
  • Knowledge in OS – Windows or Unix

Now, I know that you’re probably noticing that only the last one mentions Linux/Unix, so what does the other two have to do with learning Linux. Well, that’s the coolest part, by learning Linux, you are actually developing your skills and knowledge as a tester. Here’s why:

  • Linux is an open source OS and allows you to poke as deeper as you wish (normally until you break stuff so bad, the only way to fix it is re-install), which is a need rooted in our testing mindset – to explore things and look how deep we could inspect them. So, in the context of network protocols, you have plenty of ways and tools in Linux to learn and have a “hands on” experience and by that improve your knowledge on a topic that’s vital for testers and IT professionals in general.
  • Learning Linux, you are practically learning scripting language on the fly, because the “language” of commands you are using in terminals and consoles is a script language itself (bash, zsh, ksh, etc). It all looks like you are making yourself a great favour by learning Linux, but most importantly…

Open code means an open mind.

It’s a long topic and I won’t get too deep in it, but there’s a great valuable lesson in using Linux and learning Linux and it’s in the power of community. It opens you to the world of open source software which is huge and profound and complex … and annoying sometimes, but it could always teach you something. Many important testing tools are open source as well – Selenium for example.

I believe that the most valuable lesson that you could gain by learning Linux is how to figure out solutions for your problems on your own. And I think that’s a skill that’s really valuable in software industry at all. Because let’s be honest, when you are a rookie, dealing with Linux is such a pain in the ass. I mean, you come from Windows where everything is restricted and polite and gives you alert messages and you step into a totally non-dummy proof OS where you could practically mess up stuff so bad, that you will wonder what the fuck happened.

Then you have the Linux community, which is mostly consisted of trolling geeks that will answer to your problem with – “READ THE FUCKING MAN PAGE !!!”, like you know wtf a man page is. No, there’s cool and helpful guys, of course, but all that has its own charm, because it drives you to be more independent, to act, to search, to explore, to investigate. All these skills are really important for a tester, because we all know that if we bump into an issue, we could ask a senior or a lead for advice, but that doesn’t really develop our skills very good, on the other hand dealing with problem, finding solutions, sharpens our skills just like grinder sharpens a sword.

Natural drive to explore.

It’s a concept that was many times covered by Richard Stallman in his talks and lectures – the freedom to explore and change the insides of a tool or operating system you are using. This is not only essential for your career as an engineer, but as a tester, too. It is our distinct skill to be able to try to disassemble stuff, bring back components in different order and then fix them. We are, as everyone in technology, scientists and in order to develop, we must experiment. The best way to experiment is by learning what you are dealing with, it’s functions and components, if we don’t count reverse engineering, of course.

O, Kali.

Another great reason to learn Linux, besides the fact that 90% of the servers in the world are running it, is one really awesome skill set, that’s highly valued and really rare in software testing and this is security and penetration testing. And to cover these, there’s a great Linux distribution called Kali Linux, you can find it here: https://www.kali.org/

Basically Kali is a fully functional Linux distribution, which is loaded with any possible security, penetration, packet sniffing or social engineering tool you could think of. Of course, because security testing is so bad ass, this doesn’t come packed with a ribbon and with a nice and shiny UI, oh no. Almost any tool in Kali is a command line tool, meaning you will have to learn scripting, writing specific commands, etc. But this is definitely something that pays off, and the more you dig in it, the more tech savvy you become.

Here’s a small intro to Kali, including setup and installation on Vbox, by a fellow blogger. And here is a whole blog, dedicated to Kali Linux, for the more advanced users.

How to start learning Linux.

And after we concluded, that learning Linux is important, there comes the big question – how do we start? Well, starting is easy, just download it install it on a VM or as a live CD and give it a try. And here I hear people screaming, “but how, it’s complicated, it’s hard to use, you have to know the commands, only admins can use it…”. This is in the past, Linux nowadays has nice user interface, easy installer (Next, Next, Ok, Finish) and many other Windows like features. The thing is, sooner you forget about them – the better. Because when you have to use Linux to log into a server, there won’t be a GUI, so get used to the console, it’s nice and easy, just a bit weird in the beginning.

So, what distribution should you pick? Well, there are thousands of them, but here are my suggestions:

  • Easy to install and use – junior level – Ubuntu, Mint, CentOS – these are really user-friendly and meant to be easy to use.
  • A bit more complex – intermediate level – Fedora, Debian – these are, let’s say, the above’s big brothers, it might be a better idea to dive into them if you have previous Linux experience.
  • The elders – senior level – Arch, Slackware – these are intentionally hard to use, at least to a novice user. The reason is, they want to keep things as Unix-like as possible and as simplistic as possible. You have a lot to learn from them, because you have to do almost everything manually – disk partitioning, configuring interfaces, installing GUI, if you want one, etc. Definitely a must see, but you need to arm yourself with a lot of patience and knowledge.

Here are some resources for the novice Linux-ers:

In this blog topic I wrote about edx.org‘s course, I’ve finished it, it’s awesome and a great beginning for total noobs in Linux.

If you like a more visual tutorials – Eli The Computer Guy is your man. This guy is amazing, really good at explaining stuff in a simplistic and understandable manner, obviously knowledgeable in various OSs and networking, protocols, programming and I don’t know what more, but he has a whole section on Linux, with nice tutorials.

Nixcraft, is a great blog, where almost every problem you search for has solution, with really cool examples and explanations.

Of course, there’s a lot of paid courses and programs to learn and certify about Linux, but I am just trying to give an introduction including free and accessible materials here.

So, don’t forget to comment and share with your friends… and don’t forget to use Linux – open source, means an open mind. Good luck! 🙂

 

How to extract links from XML file using Linux grep

Recently I had offered a colleague of mine to provide her with all links to blogs I follow so she could read them, too. It was easier said than done. Needles to say, wordpress.com is very friendly in this case and provides you with an option to export all blogs you follow. You simply have to go to “Reader” > “Blogs I follow”, click “Edit” and then “Export” which appears under the text input field.

So good, so far. I was provided with an .opml file (an xml looking like mark-up) and here is where my misery began. It’s good to know, that currently I am following about 150 blogs and extract all manually would have been a punishment I don’t want 🙂  and providing an ugly .xml file to my co-worker would have been even uglier.

Solution.

I’ve done such exercises when learning C# by using regular expressions and I believe I could even do it in C, but dealing with stream readers and writers wasn’t as efficient, as I wanted. And one thing popped into my mind.

Regular expressions + Linux.

Regular expressions are wide-spread in almost any programming language, but it’s not a surprise they could be used in the Linux shell, too. So with only one command, we could do the same job as open file (with some stream), read it, write to another, and then close it in any programming language. The sweetest part here is, we don’t have to care about it, it’s all done by grep. But let’s give a some context to our task.

The markup.

This was tricky. I wont paste the actual .xml because I don’t want to have these blogs spammed, but it looked just like this:

So as we see, we are actually having a lot of information there and we have the links repeated twice, so we’ll have to keep this in mind, too.

Using grep.

Grep is command line utility that searches for a specific phrase or pattern in a text file and displays the results in another file, or through the standard output, if no file is provided.
Normally the command looks like this:

To be honest there’s no easy way to learn regex, you eighter struggle with them for long enough until you learn them, or you just apply the try-fail rountine until you get. I don’t pretend I am e pro, I am mostly using the second approach. This is the reason why I wont explain details everything you see, because there’s enough tutorials for using regex, I will just focus on our specific task.

Our command would look like this.

So before we are able to proudly say:

neo - I know regex

we must explain what we did with the command.

Ok, so the easy part is we are using grep as a command line utility and we are providing it with source file(wpcom-subscriptions.opml), and at the end we redirect the standard output to a file that will be created for us (> sites.txt). What’s left are some additional arguments that we provide (-oE) where -E stands for “extended regular expressions”, this way we could use more advanced features such as grouping, and -o which stands for “only matching”, which will redirect only the matching (otherways grep returns the whole lines, which is useless in our case).

So the only part that’s left is the regular expression itself. We start with matching the protocol “http”, than we have the symbol representing any character sign (\w) .But there’s another “hack” here, since it’s not sure if we are expecting a letter (ex.https) or a non-letter symbol (http.) we are using the pipe symbol to define one option or another. Since we really don’t care what’s coming next, we use ‘+’ symbol which will literally select every occasion defined from the previous condition, until it hit the end of the row, whitespace character (\s, \t) or the next codition.

The last part of deciphering our regular expression is the part we deal with the top-level domains of the URLs(.com, .net, etc).But first we are using the backslash (\) to escape the dot.Why is this important? The dot has it’s own meaning in regex and it’s “select anything”, that’s why we have to use it carefully, other wise we might select stuff we don’t want. Next we are using again the same paradigm – we divide them in a group, using the parenthesis, inside we list the top-level domains, we believe are present in our list, and divide them using a pipe. The last part is just cosmetic, if we take a look in the mark up again, we’ ll see that some of the URLs are ending with ‘ ” ‘ and some ending with ‘ \” ‘. We use the same pattern and switch these two options with a pipe. The last symbol – ‘\s’ represents whitespace, because our target is the link located in the section “htmlUrl” and it ends with a whitespace. If we don’t match the pattern ending with space we’ ll just end up extracting every link twice. Which we don’t want, but indeed it’s a “corner case”.

Conclusion.

It’s always exciting when you are able to use your coding/scripting skills to tackle a boring task and I hope this will be helpful. If you ever need to use regular expressions, doesn’t matter if in Linux or any programming language, and if you feel insecure on what to use in your expression, I would recommend you to use this tool. It is helpful because it provides a precise definition of your regular expression, step by step, this way you will be able to see if there some part that’s ambiguous or not precise enough.
I would love to read you opinion on this and of course if you liked the topic, feel free to comment and share with your friends.