Now, you’re probably wondering why I italicized the little quote about highly regarded sources, and if you are the foreshadowing type, you may already be able to guess where I’m going with this. First, let me say that most of the effective algorithms for extracting entities, and almost all of NLP, rely on fairly well formatted, “predictable” text. By “predictable” I mean that it follows formal grammar rules, etc. So, in selecting highly regarded sources (say, CNN?) you are constraining your statistical frame to sites that you can processes. There isn’t anything terribly wrong with doing what you’re good at, but I would like to argue that, at least at Networked Insights, we fight to keep away from this restriction. In part, formal writing carries a bias. It carries a message motivated by some latent motivations. Keeping a high reputation, for instance. It is an immensely more difficult task to harvest information from the drips-and-drabs of informal writing such as is found in twitter and forums (or even blogs).

It’s because of this that, while the thought of Evri is exciting, I don’t think it will tell you anything that you didn’t already know. Now, my goal isn’t to pick on Evri, but I think it is fascinating to realize that the reason that analyzing more formal and easily analyzed text on the web is a bit of a losing battle is because the formalism comes in part from the author knowing that we are watching. Fox or CNN write with a specific audience in mind, and that audience is the same audience that TV seeks to entertain, and on, and on, and on. What is so powerful about the social web is that it’s text produced with only an audience of two or three people being expected.

I say this is powerful in two ways. First, biases equal out by the sheer volume and diversity of publishers. If someone is trying to catch the attention of a particular audience and tailors their text to fit, then that intentional word-smithing is likely reduced by countless other authors with similar overall ideas to express, but different spins to put on the text. Second, since most of this text is being generated quickly, and admittedly not always that well thought out, the raw feelings of people tend to leak into the text. It’s the living room conversations had without thinking. It’s the reflexive “boo” at the stadium. This rawness, if you will, if far more valuable because it’s never what you were expecting.

There is power in having concrete numbers, even if they do not mean much in isolation, for at least the sake of comparison. One of the best examples of a measure without absolute meaning is baseball’s OPS. A player’s OPS is the sum of their On-base percentage (which is a pretty self-explanatory number) with the Slugging Percentage. The Slugging Percentage (SLG) is the sum of the total bases reached by a runner on a single at bat divided by the total number of at-bats that the batter has. The concept that SLG captures is the hitter’s overall effectiveness and power ^[1].

If you close your eyes and get near to that crevasse where conscientiousness and lucid dreaming meet, you may start to understand what on earth SLG or OBP meant in the minds of their creators. They are, as numbers without a context, meaningless. That said, if you give them either the context of Other Players or the context of Chance We’ll Win This Game their value is unmatched in today’s baseball. An OPS near 1 is fabulous! What does 1 mean? Well… uh… it means we win a lot, and that’s good enough (unless you are my Mets who have great stats yet constantly defy the odds). Sigh, I digress…

Software, like baseball players, is magnificent, often times mysterious, and occasionally questionably worth the sum you paid for it. And to evaluate it, like  baseball players (who’s behavior also often seems peculiar and irrational), you must boil your application down to some simple numbers you can digest.

For instance, if you were to compute a health metric to digest your system’s state, what might you use? Some insights gained from the recent software sprint I was involved in point to more than your typical CPU usage, Memory usage, and network usage. While these numbers are valuable, they do not give you a good idea as to how your system is behaving compared to how it should be behaving. That is to say, if your application uses 90% of the CPU most of the time, then that is normal and acceptable. It is not until you establish the mode water mark that you can start reasoning about what high and low look like.

To begin our application’s instrumentation we first built a restful webpage to pump our statistics out via XML. We also wired up simple serialization of all the available system JMX server named object values. This gives us, essentially for free, the max and current values for memory, garbage cleanup statistics, thread statistics, and a host of other diagnostic bits of information.

In retrospect, we should have wired our application metrics into JMX as well, and perhaps that will be a future task, but for now they reside in series of iBatis calls to the database storing simple double-precision values, their weighted moving averages, their min, max, and current values as well as some time stamping information about how long the interval is between calls to that statistic and how often the statistic has been updated.

Of particular interest is that the weighted moving average consists of only two data points: The current average and the new value. Also stored with the statistic is the weight given to the new value (the value of which should be >= 0 and <= 1). The larger this weight is, the faster the average will move toward the new value. For daily job runs this value should be high. For frequent calls to the database, this value should be low. This causes the average to move quickly for the infrequent calls and slowly for the very frequent calls.

One added bonus, errors and exceptions, regardless of language, should be treated as data to be handled and preferably captured and stored. The statistics engine does provide an exception handling routine to store an exception, though this system needs refinement as we learn more about how we will be using this information in the future.

The goal of all this a dashboard with fields not unlike the following:

• exception count for the last day
• current memory divided by peak memory
• current threads divided by peak threads
• batch job timeout count + batch job completion count = total job count
• user calls per minute

These are all instrumentable values and, at a glance, can tell you some profound things about the general activity of your system. It will tell you something about throughput and something about how close to your hardware limitations you are.

What is even more profound is that once the application holds this information, the application can “learn” and take action when the situation changes. It can throttle user queries to let the database catch up. It can send warning emails. It can stop running jobs. It can signal other clients of the DB or other resources to “back off use” for 10 minutes. These are all very simple reactions to resource strains, but impossible to do intelligently without some very basic measurements.

Share elsewhere...

People come to the internet with an established view on a particular topic. We do not care precisely what the topic is, but suffice it to say, that it is one that the Internet user has some knowledge about, perhaps would like to know more about, and has some preconceived notions about. When they come to the Internet with their question, they punch it into a search engine and get back a few hundred results.

They read the headlines and queue in on the ones that match their preconceptions. They may click on a few that are not in agreement with what they believe is true, but they feel compelled to check them out also. The vast array of disagreeing sites are as a hydra, too numerous, and in this case too varied, for our Internet-searcher to make much sense out of. In the end, they are most likely not persuaded by any of them (mostly because they are all lumped into one category of “opposing views” and the single category of “my view” has some definite, clear, coherent representation).

The result? People find what they already know on the Internet. This is how forums become so mono-thought-esque and this is why people shout-down or ignore opposing views that appear in their specialized forums. This is, of course, a generalization, but a prevalent pattern.
I point this out because it is a re-presentation of a fundamental pair of challenges that the Internet faced when it was just the World Wide Web. Social Media offers some ways to alleviate the problem, but not many.

1. How do you find a trusted voice for your point of view? That is, how can you be sure that you have a valid point of view and have found on the Internet someone who represents it with reason and balanced judgment?
2. Second, very similar to the first, and indeed necessary to satisfy the first point, how can you find balanced, well-reasoned opposition claims to the view point you are searching for? There are simply too many views that do not agree to single out the few that are perhaps actually challenging to your perspective, when considered.

To put it another way, to solve #1 you need #2. To solve #2, currently, you need to read through a lot of information and most people simply do not have time for that. As a result, they tend to form their opinion off-line and validate it online.

The result is online communities that are essentially echo chambers in which opposition perspectives have a very tough time gaining any sort of traction. Is this a bad thing? Well… not really, so long as you don’t think that your online community is some sort of truth-broker.

Social media offers some methods by which you can more easily find valid voices of agreement and opposition by creating continuity of relationships. If you can find one person with well-reasoned arguments for or against your position, chances are that they only dialog with like-quality people. If they disagree online, they most likely will do so at the same level at which they argue for their position. Thus, if you try to find valid opposition, find those who disagree with your initial social network contact on the subject matter.

Spidering along the social web, you should be able to find a much smaller set of points of view with higher quality arguments. The problem them becomes finding only that entry person, something you could perhaps accomplish in the “real” world and take online.

One last thought: Malcom Gladwell rightly points out in Tipping Point that there are different people who serve different roles in societies. He breaks them up into three groups one of which are Mavens and the other of which are Connectors. The Mavens are the subject matter experts whose opinions you want. The Connectors are social butterflies looking to bring together people. If you can find not only a subject matter expert who argues at the level you are looking at, but if you can identify who among his connections is a Connector-type of person, you will have a fabulous initial sub-network of people to read for opinions.

Share elsewhere...