Keep up the research and may the best browser win!

1. Test memory release when other apps are contending for them.

2. Loading frequently visited websites.

Will probably do them soon

1) Startup – I can definitely empathise with the issue of Firefox crashing and requiring a restart, however stability apart I don’t think restarting a browser multiple times (or requiring dodgy multiple instance hacks) should be considered “normal”. A stable browser will require none of this, and should ideally be started once, and closed once. If your browser is crashing all the time, that’s a bigger issue than performance, and perhaps you should switch browser on the back of stability not performance.

2) Again, I can empathise with the case of opening up Visual Studio (or whatever high-memory app) and Firefox not freeing it TO the newly opened app. However unless you actually try running apps concurrently DURING TESTING I don’t see how the results can be of relevance. It’s possible that the browser might free up memory to newly opened apps – you don’t know until you try it.

3) People do visit new sites daily, but the vast vast majority of what they visit is not new – people return to read new blogposts on blogs they read regularly, they post in forums they visit regularly, they revisit webmail and online services of which they are a member, etc.

So even if this statistic is somewhat relevant, I don’t think it’s very relevant. Testing how fast cached pages load would I believe be a far MORE relevant test.

3. The warm up was for the first test because as browsers build browsing histories, form filling data, auto complete, quick dial, etc, they slow down in their startup time. So, to ensure that all browsers have relatively the same usage time behind them. This will give it a more fair platform for startup.

And clearing out the cache for the last test was necessary to measure the speed for loading the pages from ground up. A lot of people do visit a lot of new sites every day, so it does matter how fast browsers are at loading new sites.

2. Memory Usage – you have a valid point here, however, I do have some past experience using FF, where I used to have it using a large amount of memory, and if I now have to run something like Visual Studio, it wouldn’t let go of memory even when I used to close almost all tabs. The only option was to close the browser. So how quickly memory is released is important IMO.

You need to remember that not every one is running awesome hardware with lots of RAM and very fast disks (not counting me). For example, this is not a problem on my desktop, but on one of my laptops with only 1GB of RAM, this becomes an issue very quickly.

1. Startup time – again a lot of people do close their browsers often enough – and I am not the only one. Or start new instances. One of the primary reasons why I shifted from FF to Chrome a year ago was because of how quickly it loaded whenever I needed to run it.

In case of FF, I used to have multiple instances of it open all the time because if one tab misbehaved it would crash the entire browser. So, my way of operating it was that I would have my key tasks open in one FF and I would keep opening and closing new instances for other work throughout the day, so the app loading speed mattered quite a bit.

Finally, at the end of the day, I would also like to fall back on my disclaimer – this is not a scientific experiment, and the relevance of results is purely subjective to every individual.

Cheers.

(1) Startup Time
I don’t know about you, but I feel startup time would be utterly irrelevant for the vast majority of users. I open my browser when I turn my PC on, I close it when I turn it off. It is by far the least commonly carried out action in anyone’s browsing time, why do you give it such prominence here?

(2) Memory Usage
You say: “how quickly does the browser release memory as pages are closed and tabs are shut down” – I wonder though, what programs are they “releasing memory” to? If you have no “competing” high memory apps running concurrently on your system, an efficient browser will HOLD ON to memory unless it is explicitly requested by another program would it not? Did you run anything concurrently for this test?

(3) Browsing Speed
This one I really don’t get at all. You say here: “For this test, I have cleared my caches in all the browsers”. WHAT? Cache is one of the primary mechanisms for browsers to SPEED UP webpage loading times. Surely clearing that out renders this entire section of the test utterly irrelevant does it not? How many users clear out their cache before each page they visit?

Above, at the beginning of the article in Setup and Preparation you’ve said: “Warmed up each browser to visit several sites just to fill up cache and cookies randomly for each browser” – what was the point of this step if you go on to clear it out again?

So, for Firefox, there were no plugins installed.