Which of these 2 antennas would work better on FM
Jpole
http://www.jpole-antenna.com/shop/low-power-fm-lpfm-broadcast-antenna/
1/4 wave
I'm assuming that you're in the U.S. If you're dealing with a Part 15 transmitter (and that is what we do here), then it really doesn't matter. You just need to ensure that your setup doesn't exceed the maximum field strength dictated by the FCC. The problem isn't getting a better antenna to go further, rather it's the other way around - ensuring that you don't exceed those miniscule field strength limits.
Most Part 15 certified FM transmitters will have built-in antennas that you can't replace. There are a few certified transmitters that don't (I own one, a Landmark FM-350, but it comes with a wire dipole that it was certified with), but more than likely either of those antennas you reference would be 'too much antenna' to stay within the rules (particularly if you ran a long coax, which would likely also radiate, to get those antennas up high). I'd also question why you would spend that kind of money, in addition to the transmitter, to have a range of anywhere between the 100 to 800 feet (depending on the receiving equipment) that is realistically achievable with Part 15 FM.
Although I'm not an antenna expert by any means, it's my understanding that the J-pole, at least theoretically, has a little bit of gain compared to a quarter wave vertical with horizontal radials (and it's roughly equal in gain to a quarter wave vertical with sloping radials or a vertical dipole).
Less likelyhood of feedline radiation with a 1/4 wave ground plane. Drooping the radials downward a bit achieves nearly a 50 ohm match. The J-pole does exhibit some gain but probably not enough for you to detect.
