I use Portable FF... not problems at all... some of the older USB drives can be quite slow.... even though they claim "usb 2" speeds.
Also if your thumbdrive is almost full.. there's no elbow room on the drive.... it may run better if you get a bigger drive or clear some stuff off.
A good way to "defrag" thumbdrive is to copy every thing to a temp file on your hardrive, then delete everything on your thumbdrive... a format is cleanest, then copy it all back on... that too may help.
Also under the "advanced" tab in FF... don't let your cache size be over about 20mb or so....it will only load down your thumbdrive.
lastly, you can go to "tools" in FF, then "addons", then in the lower right click "get extensions" when you get to the page, put in the search box "network tweak"... when that page comes up click on "Network tweak".. I don't recommend the one that says "faster firefox".. it makes your cache work in the background too much and the other one will set the proper settings for you .
Anyway... click on the link that says "install now", once its installed.. let it restart firefox.
Once it comes back up go back to tools, drop down to the new entry that says "tweak network settings" and click on the "power" button in the lower right hand area and click ok.
One last thing to can "alter" on those setting that hitting the power button makes is to:
set ?network.http.pipelining.maxrequests? to some number like 30. This means it will make 30 requests at once.
I'm at work on a protected system where most settings are locked out.. These setting are adjusting FF alone.. but it helps the browsing speed... and you don't have to worry about upsetting some network administrator at work.
Hope some of this helps.