The Dreadful State of Accuracy Accounting

Warning: Rant Ahead

I’ve pretty much had it with accuracy claims. It’s not just that some of them are extreme examples of cherry picking. It’s exceedingly rare for anyone to even define the basic variables that one would need to get an idea of what is going on.  In an age where anyone has the opportunity to better their own skills through a small amount of diligence and study, continuing on in ignorance for the sake of feeling good about oneself is silly.

First of all, and I’m pretty sure I’ve beaten this one to death already, the ubiquitous 3 round group extreme spread measurement doesn’t tell anyone anything. When I see a three round group I immediately skip it because delving any further would be a waste of time, energy, and would leave me dumber for having looked. It just doesn’t have relevance to the point that the shooter is trying to make. I don’t see 5 round groups as much better.

Secondly, I have noticed a phenomenon of willful perceptual blindness. It goes like this: “These 4 were all in a tight cluster. That’s ¾ MOA! I don’t know what happened to that fifth one over there. I probably pulled that shot.” Here’s the problem: YOU DON’T GET TO PRETEND THAT DIDN’T HAPPEN!!! It did happen, and it actually means something. If you actually know what happened when the shot broke, it may mean something else. If you were in a shooting and that magical mystery bullet hit the wrong person it would be a little harder to pretend it didn’t happen.

I expect this to happen on the internet. Where I have been really frustrated is when I see it after shelling out $8.99 for a magazine. Example: “Sniper” magazine, 2011, “Nighthawk Custom” by J. Guthrie, page 48. 5 group sizes are listed to the thousandths place. He then gives an average of the five group sizes. The load is listed as Federal Gold Medal Match 175 grain. The distance is listed as 100 yards. All that is fine. What I want to know, but can’t tell is how many rounds went into each group? That’s kind of important.  With 10 shot groups, it might be absolutely stupendous.  With a 3 round group, it might be something an average shooter with an average gun could do all damn diddley day long (yes, I said “all damn diddley day long).

It would also be nice to know whether Mr. Guthrie is giving the measurement in inches or minutes, although I can see that’s splitting hairs. He did give us the measurement to the thousandths place, so I think splitting hairs is justified. I know that he is capable of giving a full report, as he did in the 2011 Guns & Ammo Book of the AR15 article on the Les Baer .264 LBC AR, p. 36, which was perfectly adequate.  

In Issue 12 of “Recoil” magazine Erik Lund tests the Lancer L30 .308 Winchester AR. Here’s all we get: “168-grain Federal Gold Medal Match turned in .75-MOA groups—and that accuracy could be likely tightened up with handloads.” While we have no distance for the groups, at least he gives it to us in MOA. The big question, again, is how many shots made up each group? It makes a big difference.

Probably the best example I could find in the magazines I had on hand was in Iain Harrison’s review of the POF Gen 4 carbine in issue 12 of “Recoil”. It is as follows: “After being passed around like a cheap hooker and fed like a stray dog, the gun just kept running no matter what was stuffed in the mag or how fast it was emptied. And it got emptied a lot. Figuring that was enough abuse, with smoke still curling off the barrel’s exterior, I sat down at a bench with a few rounds of 77 grain OTM ammo to learn of the gun could still group. Yep. Still sub-MOA.” Wow. I loved you in Top Shot Iain, but that was not fit to be published. How hard would it have been to be slightly more specific and at least a little classy? I know the season 1 winner could handle that. Someone shopping for a $2600 rifle should probably know more about the piece than how it compares with a prostitute.  I would think that for many of us it’s not even a comparison we could understand.

There is a lot of information to be had in the world right now. There is no reason for shooters to settle for anything less than accurate and complete reporting, especially from media sources who act as de-facto leaders in the shooting community. These guys could do a lot even by picking up the free version of On Target and listing some mean radius numbers, which actually allow an easy apples to apples comparison.  With just a little more work it could be possible for a consumer of information to actually have an idea how a rifle really shoots, rather than hyperbole leading to unrealistic expectations.      

15 thoughts on “The Dreadful State of Accuracy Accounting

  1. You hit the nail on the head here. My general issue with gun journalism is that it isn’t characterized by serious, intellectual writing and all too often relies on “pop” metaphors and lingo to get its point across.
    I work in medicine, where new devices, therapies, and drugs are tested, trialed, and written about with precision and care put into the study and statistical methods used. I’d really like similar data on guns but it is hard to come by.

  2. Well, you know that most gun magazine reviews are there as thinly veiled attempts to instill a desire in the reader to run out and buy the latest ‘super gun’. Of course, most of the tests are shot from a bench. This gives us an idea of the accuracy potential of the particular firearm, but helps us very little in terms of real field performance of the rifle/operator combination.
    I guess I resolved this issue some time ago, deciding that until I can shoot my rifles with more consistency (and I have a couple of potentially sub MOA guns), I’m not going to get too excited about the accuracy claims of the new, sub .5MOA, whiz-bang, super rifles. As you mention and have adequately shown, dispersion is a fact of life with groups of larger numbers of rounds. Why is this so hard for some to grasp? As for the gun writers’ tests, we just need to know that their methods often instill unrealistic expectations in the potential buyer.

  3. Whoa -up here the unrealistic part starts when you buy the latest issue of Guns and Baloney and expect information you can actually use,these are for the most part expensive marketing and advertising pamphlets ,that plus the “my gun will shot sub-Min at 1000 yds all day long” crap being peddled on net, it is truly a wonder there are any pockets of sanity left in the Rifle shooter world. I know I’m old and cranky,but you pushed a button bringing this sorry state of affairs up- makes me rant

  4. The lack of a statistically valid, standardized accuracy test is pretty ridiculous, especially when you compare it to other hobbies (you don’t expect to see some weird non-standard dyno chart in a car mag, for instance).

    My guess is that they don’t want to make fine distinctions when 95% of the population (myself included) cannot shoot up to the gun.

    • Where it gets me peeved is that “sub-MOA all day long” plus “modern guns are more accurate than the shooters” equals “any dispersion over all the shots going in the same hole is the shooter’s fault”. Maybe, maybe not. Most guns with most ammo, and more than a couple of rounds in a group WILL begin to exhibit some dispersion. Now when I hear people second themselves at the sight of a stray round I can’t help but think that they’re denying a completely normal phenomenon.

  5. I don’t mind 5-shot groups for a deer/elk rifle & load, but I will shoot several of them to get some idea of consistency/repeatability. For more stringent accuracy requirements, mo’ be better.

  6. “… it’s not even a comparison we could understand.”
    That be true, but you did understand that those publications are normally running on ‘Sex Sells’ and equating shooting with lack of something in most men – a compensation that will never disappear until the last of the anti-gunners goes.

    And I just finished reading an author tear up a whiner about self publishing – same story as yours. The gate keepers (publishers) want their out come, not the readers nor even the writers. They count success as number of buyers, not the number of word of mouth referrals. Hogwarts succeeded on word of mouth.

    What you seem to want is solid information that you would produce, if you had the opportunity. Or the results that would be meaningful to your mind about the rifle and the ammunition and the shooting. In the world of superlatives, you want truth, and are certain that all isn’t equal nor should it be… thought so or painted that way.

  7. I am more interested in techniques (whether field shooting or handloading) than new rifle and gadget reviews that most magazines and blog sites seem to focus on these days. Your blog and Colorado Pete’s two books are both a breath of fresh air in a dark smelly room…………

    Beyond technical articles, what happened to the great historical and hunting stories that gun magazines used to have???

    • Thanks for the kind words Richard!

      I think those great stories died with Peter Capstick and Finn Agaard, who lived them.

  8. Yip, on the nail.
    I guess they are working on a statistical 85 percentile standard deviation model… which would actually probably be about right. Too bad the additional 15% actually tells you more about your rifle and shooting habits than the 85%.

    • Rob,

      Would you be willing to elaborate on your point? When I hear something I don’t understand I get excited that I might learn something.

      • The idea with the standard deviations is that you can use statistics to model the shooting performance and then estimate the probability of future results. I work in clinical research and we use a “95% confidence interval” around our predictions. So in this context, we might apply statistical testing methods to how we measure the rifle performance, and then be say that the rifle’s MOA average was 0.9 with a 95% confidence interval of 0.6-1.2; meaning that if we were to repeat the test, 95% of the time that average will be between 0.6-1.2. Applying rigorous statistical methods is not something I’ve ever seen a mainstream gun reviewer do. It’s not sexy—it’s actually really geeky. And honestly, I don’t think the market cares that much about it. How many people can actually outshoot their guns? I can’t consistently shoot sub-MOA.

        • Thanks Samuel… that’s pretty much what I was intimating.
          My musing was more to do with how many bullets do you have to send down range in order for the centre POI to actually be relevant, and at what point is a ‘flier’ part of that 95%-100%, or are the first 4 shots of your neat little 5 shot statistically .2mil off centre and the flier actually on the other side of the POA… and had you shot 10 more shots, would they have gravitated to the flier, or the remainder of the yet to be completed group.
          So unless you know that the flier was YOUR fault, i.e. flinch, pull, etc, then it should remain in the statistic even though it pulls your POI away from your group.
          If you have a look at the WEZ analysis that Bryan Litz is doing, it’ll make a lot more sense, but his virtual ‘gun’ literally fires a 1000 rounds to create the patterns that he predicts.

          • Statistically, I think it would be hard to really make any kind of conclusions with fewer than 30 shots. I hadn’t seen the WEZ stuff before and find it really fascinating.

            Personally, I’d love to see gun reviews that had 10+ shooters with a known, statistically calculated “handicap” (like golf) shooting the gun in a variety of scenarios and providing analysis like this blog does. But I think not many of those reading gun magazines would be interested in this.

Leave a Reply

Your email address will not be published. Required fields are marked *