I wasn't going to speak to this again, but I've had a lot of issues niggling at me, and they finally coalesced today.
First, the shootout wasn't science. Now, before anyone goes all crazy over this statement, hear me out.
Science is a methodology in which you first form a hypothesis, then collect data in an effort to disprove it. If you can't, then the hypothesis MAY be correct, at least until more data comes along. You can't PROVE anything, except perhaps in mathematics.
No, the shootout was pure and simple data collection, and from a limited sample size of 1 for each of the transmitters. That was freely admitted.
It's quite possible, and in fact, highly likely, that the results for the AMT5000 were compromised. Because it would have been my hypothesis that the Rangemaster and the AMT5000 would be relatively similar in results, and lead the pack in terms of efficiency in delivering a signal to a Part 15 antenna (I'm not familiar with the Grain transmitter).
The fact that the AMT5000 performed relatively poorly could be from a number of factors, particularly since it's constructed from a kit, where quality control of the final product is unknown, and any number of things could happen to individual parts during the assembly process. I've owned 4 factory assembled ProCasters, and their individual performances (in the same installation) have even varied widely.
But instead of attempting to determine the reasons why, the single sample data has been published on the Internet and a magazine. No one is likely to read the caveats associated with it - they'll just take away the results (I'd be surprised if even 10% read the methodology used).
So while the limited data available is interesting, and a start, it's hardly the final word. And hardly science.
One last thing has also pestered me about the whole shooting, so I will make it my parting comment on the subject.
The grand announcement was cleverly made con-currently on two part 15 websites, owing to clever planning and co-ordination.
For that brief moment in time Part15.us was important and expanded the total outreach beyond the originating website. This was allowed by the good grace of Part 15.us and there has been no objection to being utilized as an adjunct publicity outlet.
However, before and since we are demoted to the status of the "kiddie site" and otherwise flamed with such sly remarks as "What other site?"
Advice: respect us or disrespect us, but don't flip between both.
If you straddle both sides of the fence, you are bound to get your ___'s hung up in the barb wire.
We are taught that the scientific method begins with a hypothesis but this is not the way it actually works. Scientific study begins with an observation from which a hypothesis is formulated and tested. The standard tests involve mathematical statistics which are based on probability and the necessary use of probability is the reason that something is not definitively proven or rejected. For example, we cannot say that someone who has high blood pressure will become ill, only that there is an increased risk of illness which can be assigned a probability which only applies to the whole population and not the individual.
In engineering and technology things are different because probabilities and scientific study are not needed to predict the operating characteristics of a device. The performance is determined at the time of the design and not "discovered" after the fact and comparison tests are not science and need not be. The transmitter comparison is a tabulation of descriptive data not inferential data based on statistics as would be the case in science.
If two transmitters have identical specifications and do not produce comparable results then this is due to some difference in the devices which may not be known or which the specifications do not address. For example, if both are designed and verified to have an input power of 100 mW and properly tune to and match an antenna system then a difference in field strength could be due to the unspecified losses in the loading coil and tuning network. A claim of a certain efficiency at the output stage neglects the loss between this stage and the antenna system and therefore the specifications needed to predict performance are incomplete.
There is no science involved in measuring the loss between the final and the antenna system and no science needed to account for this in explaining the differences in performance. The next step, if one is interested, in explaining the difference is to make this measurement and if this is not an explanation then look for some other difference. This is how designs can be improved.
In both science and descriptive studies it is vital to keep the results in the context of the experiment and not extrapolate beyond this. In medicine this mistake is made frequently where it is assumed that a treatment which benefits a sick person will also benefit a healthy person as prophylaxis but in technology there is a definite disconnect between the findings of an experiment and projecting these to generalizations. What prediction can be made about the comparative performance of two transmitters from the experiment when both are immersed in water? None!
There is nothing fundamental which invalidates the experiment and categorizing it as not being science is accurate but not disqualifying.
Neil
What you are describing is all right as far as it goes, Neil. But you neglect to mention that the end results are being published as a 'Shootout', i.e., a contest, with a winner and the implicit hypothesis that the #1 transmitter is the best. Which is hardly accurate, considering the small sample size. That isn't science and it isn't engineering either.
It wouldn't have taken much more time or effort to have a sample size greater than 1. Even 2 would have been far more satisfying and made the results more compelling. It makes me wonder how much real life experience the individuals who devised the tests really have with Part 15. I'm hardly an expert, but have used and tested a large number of the available transmitters for almost 10 years; and yet as I've posted previously, I've seen wide variations in range for individual transmitters (same model) that have been installed identically. I've grown to believe that there are other factors at work that come into play because of the miniscule signal being generated in Part 15 (as compared to higher powered transmitters). Not proven, of course, but just from empirical evidence.
I've grown to believe that there are other factors at work that come into play because of the miniscule signal being generated in Part 15 (as compared to higher powered transmitters). Not proven, of course, but just from empirical evidence.
Would you elaborate on that? I not sure what you're saying.
Sure.
Time of day, for one.
Now, I recognize that it's not really time of day, but atmospheric conditions associated with the time of day. I've noted from many years of listening to my own signal that I get greater range early in the morning, when it's cooler and drier. Perhaps it's also the dew improving grounding conditions. Warmer and humid conditions seem to lessen the range (perhaps even pollution).
Later in the day I've noted my signal drop, and found that another station which had not been there had suddenly popped up due to skip conditions (and it certainly wasn't night). It doesn't take much interference and background noise to get in the way of a Part 15 signal.
So, if you only use a sample size of 1 in an experiment of this sort, these effects are going to be buried.
A more representative and fairer experiment would be to use at least 3 transmitters of each type, and test them out at various parts of the day (so you get a reading for the Rangemaster at the start, middle and end of day, the AMT 5000 similarly, etc.). Take a mean or average of the various results and I believe you will get a lot closer to the truth.
Niel, I will print out your definition of "what is science," because I think it's not well understood even by those of us who imagine we are being "scientific."
But we also hear that broadcasting is an "art."
Please make a better description, but I would call art "doing a good job making it look easy."
I try first for art, and if that flops I give some thought to science.
Science is: Hypothesis -> Design Experiment ->Data Gathering -> Conclusions -> Revised Hypothesis -> More Data Gathering etc. until Conclusions (either you've disproved the hypothesis or it's the best at this particular moment with the information at hand).
The shootout was relatively simplistic data gathering only (I'm not minimizing the effort, it's obviously the first such exercise to compare multiple transmitters and significant time and money went into it). But, at the end of the day, it's so simple (i.e., a sample size of 1) that you really can't draw any conclusions from it other than with that sample size of 1, and with one experiment, and on that particular day, here are the results. THAT'S what I believe is behind a lot of the unease with these results, although perhaps more personalities than necessary have been put 'out there' in the discussion.
Maybe if more people with different viewpoints had been involved in the design of the experiment (i.e., a real shootout), some of the deficiencies could have been corrected and we would have had a more rigorous outcome. But then, now these guys don't even want to let anyone other than their little 'club' in to read their Forums, and get access to the online version of the magazine article, never mind letting others have input to what they obviously felt was THEIR exercise.
It's too bad.
So I propose an idea.
We all have transmitters and antennas of varying types, makes, versions, etc.
None of us (that I am aware of) run critical systems with those setups.
Why don't we gather, physically, somewhere central to all of us, for a few purposes.
Primarily, we should get together for fellowship.
Also, we could have a workshop to help new people understand how our stuff works, and we could also put on our own shootout. There has to be someone who owns the calibrated gear to do this for real. And since most of us aren't jerks to the manufacturers, I would bet we get a better sampling. Zero bias. The best person qualified to configure a specific transmitter should be the one who does that.
Also - if you stuck us all together in one spot, I bet we could come up with some awesome solutions in realtime to some recurring issues.
And; I am certified in 2 different circuit martial arts tournament organizations as center judge, so if stuff gets nuts, we can put the gloves on and go at it 😉
mir, you must be talking about the ALPB (the Association for Low Power Broadcasters) which is now over a year old and meets the first and third Saturday of every month at 9 PM Eastern Daylight Time on TeamSpeak.
To join contact [email protected]
We have fellowship and hold discussions about everything including food health and jukebox rejuvanation. We also hold mini-seminars on technical and programming issues.
Of historic interest, when the new owner, Jon Paul Janze, took control of part15.us, he asked if anyone would be interested in writing transmitter reviews for publication here, but no one was inclined to take up the challenge.
I ran comparative tests between the AMT3000 and AMT5000, but had no meaningful field test equipment.
Member kc8gpd does have a FIM, but he just quit the hobby day before yesterday.
Perhaps KC8GPD will be liquidating that piece as well.
He might, however, it seems he generally applies premium pricing on his many for sale items
WOW.. $600 a week?
It's more likely that we get Bill to loan us his..