Monday, March 23rd, 2009 at 11:51 am  |  13 responses

Tweet

Changing the Stat Quo

A Q + A with 82Games founder Roland Beech.

by John Krolik

I am a huge fan of 82games.com. If you consider yourself a real fan of basketball and haven’t been to 82games yet, do so right now. For years, 82games has provided data to fans that has long been misconstrued or only given in anecdotal evidence, and last week I was able to interview its founder Roland Beech. Here’s a transcript of our conversation, which you’ll hopefully enjoy.

SLAM: First off, the basic stuff: How did 82games get started up? What was the original mission statement, if there was one? Has it changed at all?
Roland Beech: I had worked on NFL stats for a while and I guess saw an opportunity to do some interesting things in basketball, which is a game like football that has a lot of interaction between players. The goal was to get involved with actual teams and so in some ways I guess the site was a bit of an online resume…

SLAM: Who does all that game-charting for you guys?
RB: I have access to a lot of different resources and then there are people charting games for additional detail, much of which doesn’t actually show up on the public parts of the site. On some of the little oddity things we’ve charted it’s been a huge number of people.

SLAM: As a creative writing guy and someone who finds math quite terrifying, one thing I like about your site as opposed to a lot of APBRmetrics places is that there are really very few formulas on the site—it’s mostly a combination of curiosity and data most others don’t have to put values on things that have long been assumed, like what guys shoot on midrange jumpers, how many of their baskets are assisted, and the like. Do you consider 82games an advanced-stats place, or do you think you’re doing something different?
RB: Right I kind of intentionally present most data in really a raw, unadjusted form. Anytime you make adjustments you are using some kind of assumptions, which may or may not be true. I do think the site is ‘advanced stats’ but I’m much more of a ‘let’s collect more data’ type of analyst rather than delving into trying to infer things through regressions, etc.

SLAM: That said, there are formulas on your site—how firmly do you believe in them? Does the “Roland Rating” represent the guys, to the letter, you think are the best players in the NBA? Do you think the best clutch players in the NBA are the guys with the best “Clutch” stats? Are these formulas closer to mathematical proofs or just another tool?
RB: I am not a fan of one number, overall type player ratings since I don’t think players have constant value. Their contributions depend heavily on who they play with, the coaching schemes, the role they are asked to play, whether they are happy, healthy, etc. The Roland Rating used to just be straight on/off but then people started to think I was advocating that as a stand alone player rating, so I added in a few more simple elements, intending maybe one day to publish a more comprehensive rating system, but that hasn’t been a priority since I don’t really look at players in that way. On the other hand something like ‘clutch stats’ is a pretty straightforward look at some specific numbers and so yes, I’m happy to say that a player is a good clutch scorer or something by stats.

SLAM: Does it frustrate you sometimes that you have this wonderfully uncomplicated data available, for free, in plain sight, and people still choose to ignore it and use anecdotes to fill in gaps in their knowledge your data covers? Joe Morgan became a poster-child for willful statistical ignorance in baseball. Please don’t call out any media guys because I have to share $7 media dinners with them, but what are some things you hear announcers or columnists write or say all the time that just drive you insane?
RB: Not entirely in that I’ve done very little to market the site. Basically once I started working for a team the site in some ways just plods along without a lot of the more innovative things I’m doing now being visible. At the same time I am frustrated when people hold opinions which are clearly refuted with stats that to me seem readily available.

SLAM: There was an anecdote in Pistol about a post-Maravich LSU coach going back to the play-by-play data of all Pete’s LSU games and seeing how many threes he would have hit per game based on how many 21+ foot jumpers he was making. Is there retroactive data that exists like that for the NBA for people that would want to know? For example, what Michael Jordan and Larry Bird’s eFG% on jumpers would have been, or what percentage of Dominique’s shots were dunks? If there is, would you just need a lot more interns to record it?
RB: It will be a while I think before we will get any full detail on the eras before say 2000. That will require some group to make a concerted effort to go back and track all kinds of data that wasn’t being done at the time. For example, we don’t even know how many shots Bill Russell blocked since the NBA didn’t track that back then. By the same token, eFG% can be calculated from box score info.

SLAM: I know you guys have some higher-level data that you provide privately to pro teams—how many NBA franchises use your service or a similar type of service?
RB: I work for one team exclusively, and have done so for a number of years.

SLAM: Could you maybe even just drop a little hint about what the types of data the public can’t see would be?
RB: It’s a lot of additional detail on what’s happening during the games. For some reference Synergy certainly carries a lot of things that look at player traits, team play types, etc. What we do is also along the lines of tracking performance with the nuances, for example using a screen, driving into the paint, who is guarding who on a play, etc.

SLAM: On that note, there seem to be two general schools of thought with regard to how APBRmetrics will work moving forward—there’s a more box-score based metric and a more +/- based school of thought. PER seems to be as well as anybody’s done in trying to make what you see in the box score a sort of all-encompassing stat, and Hollinger himself will tell you it’s far from perfect and he uses it more as a tool than an end-all. Meanwhile, David Berri and the Wages of Wins guys took a crack at using conventional stats to explain why teams win and lose, and the consensus seems to be they missed their intended mark by a fairly wide margin and ended up coming to conclusions like shot creation isn’t a skill and all credit for a defensive stop goes to the rebounder. I’d imagine the brass ring of APBRmetrics is to find something like PECOTA in baseball, which predicted not only which players would do what but tied that production into an explanation of team success, and Nate Silver ended up doing worlds better than any conventional “expert” at predicting what would happen in baseball last year before moving onto bigger things. Is it possible to find a PECOTA, Win Shares, or Equivalent Average-like formula in basketball from the numbers we have, or is it a fool’s errand?
RB: Well I am not after the one number rating, neither is Dean Oliver and a number of other people. Yes the stats folks often tend to be people using box score data only since that’s what they have at hand. Similarly there are a number of folks who are ‘true believers’ in regression based +/- type metrics. I simply feel we need to go out and collect more data on the specifics of games and that when we have this data things will be much more self evident. This is already happening. For example, a lot of people point to defense as one of the missing ingredients in the box score, but by tracking who is guarding who on plays and what transpires you can actually create very detailed defensive stats, and then even adjust them by the quality of player being guarded, etc. There’s no need once you have the data to try and deduce things, it’s right in front of you.

SLAM: Let’s talk about the other “half” (I’m oversimplifying) of APBRmetrics, the +/- data that really started with you guys and has really picked up steam since that Michael Lewis article, which made Shane Battier into the geek hero that Jason Collins never quite became. I found it interesting that Battier became the focus of the article, because perimeter defenders have always been guys no stat could find, not even opponent PER, since they’re always guarding the best scorer, or +/-, since they’re coming in when the best scorer is. There are obviously some holes in both your kind of +/- and the NBA.com kind, but it fills in a lot of what were previously dark areas, and I know some work is being done with adjusted +/-. Is this what we should look for as the player evaluation tool of the future and what’s going to get the brass ring? Or does the fact that the “Roland Rating,” which bears your name, gives equal weight to +/- and PER answer my question already?
RB: I like to include team influence numbers in any kind of player evaluation and that can be on/off, a simple adjusted plus/minus (not regression based) and so on. Yet I don’t think you ever want to fully rely on only those kinds of things—it’s just part of the puzzle. Oddly while I have published a lot of regression based ‘adjusted +/-’ articles on 82games, I am not actually a fan of that approach. I think again, with more data on hand you can really understand a player’s strengths, weaknesses and traits very clearly without having to resort to mathematical techniques to try and extract info that you think is ‘missing.’

SLAM: What’s the most surprising thing you guys found when you guys had accumulated enough game-charting data to have significant findings? (For me, it was how inefficient mid-range jumpers are across the board.)
RB: Yes I think the mid-range shots being so low a percentage for so many players is a key finding (when you consider it’s rare to draw a foul on those shots, the offensive rebound rates are not good, it’s even worse!) and something that we may see become more apparent in team strategy going forward. A team like Orlando is certainly upping the ante on a 3pt/inside skew, as has D’Antoni for some time in both Phoenix and New York. Likewise despite all the myths of ‘super clutch players’ the actual league-wide performance in game-winning shot type situations is startling low (about 30 percent).

SLAM: Do you think your site changed the way the game is played on the floor? Do you think it will? For example, at the recent SLOAN conference, John Huizinga presented some fairly definitive evidence against the “hot-hand” theory. We know Daryl Morey believes in this stuff; he hosted the conference. So how long until coaches and players start buying in and we see Ron Artest holding back on a contested jumper after he’s made two in a row? (By the way, easily my favorite part of that story is picturing Morey going up to Tracy McGrady and explaining that his 13 points in 30 seconds was a “Black Swan.”)
RB: I think there is already influence in the front office type decision making for a lot of teams, and I know there are indeed a lot of coaches who are quite into stats, although still a lot who don’t and in some cases have such accumulated wisdom that stats may not be helpful for them. At the same time, it’s not always easy to communicate to a player some piece of insight that is going to help them. So it’s an ongoing process. Obviously if teams known to be using more statistical analysis have success that will probably hasten the acceptance and usage, whereas if say Houston flounders that might be used as ammunition against stats people playing more significant roles.

SLAM: As all-explaining as baseball statistics have become, for a long time their blind spot was defense, which has given APBRmetrics guys some fits too; PER doesn’t account for it at all, and everyone knows counting blocks and steals are just as incomplete as counting errors was in baseball. +/- does some great things for accounting for defense. How close are current defensive statistics to offensive ones?
RB: Well the readily available defensive stats that incorporate team level performance are useful already and the more detailed charting that is perhaps not so freely seen is able to also take us a long way. In a few years I don’t think anyone will be talking about the difficulty in measuring player defense…

SLAM: Are there any more holes in stats that can be filled in the near future? I think assists are woefully incomplete for rating passers; not only does a chest pass from the top of the key to Ray Allen curling off a screen for a contested three count the same as a behind-the-back feed for a dunk after slicing through the defense (although your new “passer ratings” take that into account), but assists leading to fouls count for nothing, and, as Laker fans will tell you, a “hockey assist” doesn’t exist on the box score. Are these the questions that sites like 82games will be able to measure in the near future, or are we just going to have to continue guessing and ultimately deciding that we need to watch every play to know how truly good players are, like football linemen?
RB: Right, assists are probably the most biased box score stat and make little distinction between a ‘heavily assisted’ and ‘lightly assisted’ scoring chance. By tracking every pass you can obviously do a lot more interesting things with passing stats like fg% off a pass, pts/poss, breakdowns of specific passes (location to target), player patterns, and so on. This is an area that is easy to ‘correct.’ Whether what we’re doing now comes to light more on 82games publicly depends quite a bit on whether I continue to work for a team under the guise that it’s more useful for one team to know something than to dilute the value, or elect to move away from that to making the site a more significant public resource force.

SLAM: Finally, a lot of people think stats guys are robots who hate sports. Explain how APBRmetrics have helped you love basketball more.
RB: I certainly like NBA basketball a great deal and for me understanding the game and the players better typically only furthers my enthusiasm, but the truth is many fans probably get more enjoyment out of being somewhat ‘in the dark’ as to NBA realities and that’s fine. It’s ultimately entertainment. I take issue with the notion that teams should be all about a championship or they need to blow things up. It was sad to see the Suns dismantled prematurely to my mind when they were such a great team to watch, and had certainly some significant success with still the hope of finally breaking through.