This problem has actually happen quite a while for me. Everytime I plug in FirefoxPortable in my office PC, it will automatically install Java Quick Starter 1.0 extension on the first run. What annoy me the most is, this extension CANNOT be uninstalled from Firefox Extension menu, and it is NOT located in the extensions folder in Firefox Profile.

After googling around, I found out this post at MozillaZine forum. It seems that this situation happens after I updated my JRE to Version 6 Update 10 a while back. By default, after installing JRE6u10, Java will install the Java Quick Starter extension to your Firefox through Windows Registry. So, if you want to get rid of this extension, follow this steps:

WARNING!!! Do NOT perform these steps, unless you REALLY know what you are doing!!

1) Open up Windows Registry.
2) Navigate to "HKEY_LOCAL_MACHINE\SOFTWARE\Mozilla\Firefox\extensions".
3) Remove the entry named "jqs" (or something similar).
4) Restart your Firefox.