Logistic regression in individual test scoring by Chris T

If you've read any online reviews in the last, say, 10 years—you probably look for that bigass number in bold at the top and move along after seeing it's not a 10.

Some outlets are moving toward a system of objective scoring, but problems arise when limited datasets and auto-scaling creates anomalies with product rankings. Sometimes that 10/10 really is meaningless.

That's why philosophy is so important, and I'll cover just one of the ways I look at metrics here. 

For any test that can record results that rely on human perception to determine if something is "good" or "bad," you may want to use a logistic regression instead of a linear one or relational model in order to properly score a product. The truth is, there are lots of products out there that have wildly inflated scores on some review sites based on mathematically irrelevant differences in test readings. The truth is, humans aren't going to be able to discern the difference between screen black levels of 0.003cd/m^2 and 0.002cd/m^2. Similarly, it won't matter if a smartphone's peak brightness is 1,000cd/m^2 vs. 100,000cd/m^2, so it makes no sense to score these against a linear model—lest all other scores be rendered "shit" compared to a ridiculous outlier of zero utility. 

The benefit of a logistic regression is that we can set limits for scoring at the average human limits, and award points (or take them away) in an exponentially decreasing fashion. Thus, something that's slightly better than anything a human could see gets a 95/100, and something that's right at the limit gets 90/100, In the brightness example above, that outlier would get a 100/100 and push the "decent" result to 1/100 in a normal relational auto-scaling model. Instead of relating all scores to each other, we weigh the results against what someone could actually experience instead.

Let's look at screen brightness.

Brightness score (Regression)

X = screen brightness (cd/m^2), Y = score/100

Using an equation (f(x) = 100/1+200e^-(0.009*(screen brightness in cd/m^2))), we can make a chart that shows what the limits could be.

Looking at the chart, we can see that the inflection point is around 350cd/m^2 and the crest is around 850. That's no accident: 350cd/m^2 is the minimum brightness needed to see an image in direct sunlight. 800cd/m^2 is the threshold of pain in a well-lit room. While these aren't scientific limits, they're just the ones I chose for this illustration. 

Note how the score doesn't reward ludicrous screen brightnesses past a certain point? See how the algorithm keeps sub-350 readouts in the scoring basement? That's by design. We set acceptable limits for what people need based on the philosophy of the product. By establishing the necessary parameters, we can then score against them.

We also don't want to discourage reaching for better heights, so we do reward the brighter screens—though we also want to preserve the philosophical integrity of the system. If we were to score our original hypothetical, 100,000cd/m^2 would get 99.9/100, the 1,000cd/m^2 would get 98/100. Both are ultra-high levels, but one is completely ludicrous, the other is realistic.

The user is the object of the product, not the scoring algorithm, so replacing scoring that rewards beating the pack with scoring that observes the user's needs is the right way forward.

Flexing my brain... might. by Chris T

So I'm extremely happy at AndroidAuthority. Wanna know why? Because they let me be a damn expert and help educate rather than gloss over critical concepts.

To that end, I've been having a ball writing explainers on concepts relating to personal audio and imaging. Here are some of my best lately:

But I gotta say, my two standouts have to be How smartphone cameras work and What is isolation.

That's it for now, but more on the way!

—C

Back in Tech Land by Chris T

Holy hell it sucks to work for yourself.

After freelancing and working for myself for the better part of a year, I'm back with a steady gig, this time at AndroidAuthority! Well, SoundGuys for now, but that may change as time goes on.

I'll miss cameras, but at least I'm back into doing product photography, and I'm much better at video work now. I wish my D600 hadn't bit the dust with a shard of its own mirror, but whaddaya gonna do. Anyways, onto content:

Roundups:

Reviews:

 

  •  

Man I am awful at regular updates by Chris T

So as you've probably guessed, I've been busy (shocking, I know). Here's a brief roundup of stuff I've published in the last year. I added a section of portrait photos on the front page, and I've been pruning/replacing images in the other galleries to reflect my more current publishes.

Headphones:

Planar magnetic cans are awesome, and Audeze's Sine are cool as hell; expensive, but not as bad at the AK T8ie. However, you can never go wrong with an old favorite like the ATH-M50x, or maybe a wireless set of buds like the Decibullz Wireless

Cameras: 

I spent most of my time this year on imaging, so this one's gonna be a long segment—get a bag ready. First the awesome: Sony's A7R II is incredible, and Panasonic's GX85 with dual-IS is absolutely insane, along with the higher-end GX8. Panasonic's G7 is also pretty decent, but lags behind the GX85. Canon's 5DS is a megapixel monster, and Fuji's X70 is one of the all-time great point-and-shoots.

I reviewed a few disappointing models too, including the Canon EOS 80D, and the Canon EOS M3.

Getting back to point-and-shoots:

 Nikon's AW130 still rocks, even after a year.

Nikon's AW130 still rocks, even after a year.

Laptops:

I took a deep dive on Chromebooks and who they're a good choice for. Not my usual format, but I think it worked fairly well.

ProtoMulti.media

I got all the paperwork through, and May 1st was the 1st anniversary of my personal company, ProtoMulti.media! I've had a few clients so far, but it's nice to have.

 

That's it for now, but I imagine I'll be publishing soon. You can always follow me on twitter at @cthomastech for more timely updates.