Test server 25-Jan release

The 25-Jan-2008 build of the Test server has a bunch of new combat log messages available to help track accuracy. You can see the patch notes on the official forums. What this means to HeroStats is that it's now much easier to track accuracy of a power, and it's even possible to see the accuracy of an enemy's powers!

The latest unstable build (download it from the unstable web site) takes these new combat log messages into account. You'll see a bunch of new statistics related to accuracy if you're running the unstable build on Test. Here's a brief rundown of what's new:

• Both player and enemy accuracy is tracked. Each section discussed below comes in two flavors, one for player stats and one for enemy stats. For players, everything is tracked by power. For enemies, it's tracked by both enemy type and power (since a Behemoth's version of Fire Sword might be different than the Fire Thorn Caster version of Fire Sword).
  
• [Player/Enemy] Accuracy: This section measures the actual accuracy for each power over the time you've been collecting statistics. It's calculated from actual hits / (actual hits + actual misses).
• [Player/Enemy] Hits and Misses: A detailed breakdown of the number of hits and misses per power.
  
• Hits and Misses are simply counted from the new log messages.
  
• Hits (calculated) is generated by looking for messages where the roll is less than or equal to the percent chance to-hit. That's a debugging tool, mainly - in theory Hits should always equal Hits (calculated) - if it doesn't then something's not right.
  
• Number of Rolls is the count of times you had to make a to-hit roll. You might think this should always be equal to the number of times you attacked, but it's not always the case - sometimes a power is auto-hit, and sometimes the streakbreaker gets involved.
• Streakbreaker Count is the number of times the streakbreaker was used to force a hit. I'm not going to try to describe the streakbreaker here, Arcanaville's Guide to Defense does it better than I would.
  
• [Player/Enemy] To-Hit Chance: The to-hit chance percentage from the new combat log messages.
• Average is exactly what you'd expect, the average of the percentages from all the hits and misses for the power. Note that auto-hit and streakbreaker "hits" aren't counted.
• Total isn't all that useful, it just adds up all the to-hit numbers. It's used for calculating the average.
• [Player/Enemy] To-Hit Rolls: The to-hit rolls from the new combat log messages. This is the same as the to-hit chance above except it's the roll numbers.
  

The old accuracy and hit/miss stats are still in HeroStats for now. Once these new combat log messages are stabilized I'll probably remove that, but for now it's interesting to see how the old and new ways differ. From the little testing I've done so far, they're pretty much the same, which is a good thing.

All this information is accurate as of the 25-Jan-2008 Test build and the 3.9.9.8 version of HeroStats. New Test builds and new HeroStats versions could be a bit different, keep an eye on the change logs for each.

HeroStats, Real Numbers, and City Vault

With the new year, NCSoft has announced several upcoming features for City of Heroes/Villains in their 2008 State of the Game article. There's a lot of good information there which I encourage everyone interested in CoX to read. Here, I'd like to address a couple of things of interest to HeroStats users.

The Real Numbers

As a general quality of life improvement we will be adding new functionality in a coming version of the game to display the “real” numbers behind your characters and their powers. This includes a user configurable display of important characteristics such as movement rate, regeneration rate, resistances and many more! Additionally, you will be able to see how those statistics are affected in real time by buffs and/or debuffs affecting your character.

The article mentions the ability to display the actual values for the various statistics that your character has - regeneration, resistance, movement, etc. Let me first say that I think this is long overdue, and a welcome change. I remember this request coming up way back in City of Heroes beta, and it's been a regular discussion topic ever since. It's great to hear that it's coming.

One of the core functions of HeroStats has always been to track the actual values during a play session for the accuracy of your powers, damage dealt, damage received, etc. Note that the article didn't mention any of these sorts of statistics. It's possible they may be included, and that would be wonderful, but I have my doubts. As long as this sort of information is not available through the game client, HeroStats will continue to provide it.

What was mentioned in the article is the type of information you see in a build planner. How much resistance to smash/lethal do I have? What's my total regeneration? Those sorts of statistics have never been tracked by HeroStats (because they don't show up in the combat log). It's great to see that this information will be available without having to put all my information into an external build planner tool. (Side note - build planners will still be useful, though. There was no mention in the article of any way to do a "what if" scenario, and that's where the build planners are most useful.)

There's one possible area of overlap - recharge timers. If the UI is going to be able to display how much time is left before a power recharges, the recharge timer function of HeroStats may become obsolete. This wouldn't break my heart - recharge timers are less than completely accurate, as I've described before. If the developers want to change the game UI to do a better job, more power to 'em.

Here again, however, I have my doubts. There was much furor on the forums about the UI change to have buff timers blink 10 seconds before expiration, and how it would make HeroStats buff timers obsolete. Hasn't happened - plenty of people still use HeroStats for tracking buffs, because 10 seconds just isn't enough warning much of the time. We'll have to see exactly how the UI changes are implemented before making any assumptions about what HeroStats functions will become obsolete.

City Vault

We gave a sneak peak of the new “City Vault” feature at the recent San Jose Meet ‘n Greet. The City Vault is coming soon to the City of Heroes website and will showcase the millions of interesting Heroes and Villains found in Paragon City and the Rogue Isles. It will allow you to search for characters by a number of different criteria, and then view those characters, including images of different costumes, powers, badges, friends, stats by zone and other fun information! As we have previously announced, you can manage what information will be displayed through the City Vault for your characters via several in game options.

Again, let me first say that I'm very happy to see this. The ability to see in-game info outside of the game has been a much-requested feature for a long time. It's great to see it becoming reality.

None of the functions mentioned in the article will impact HeroStats directly. The core function of HeroStats, tracking statistics within a play session, is in no way impacted by the ability to view character information on the web.

HeroStats does, however, have some secondary functions that could be impacted by the City Vault. Several community websites (see our affiliates list at http://herostats.org) use HeroStats to gather information about characters. Much of this information may now be viewable through the City Vault.

We'll have to wait for the actual City Vault to be available to know exactly how it will impact these community websites. At the moment, I'm willing to say that it's highly unlikely that the City Vault feed will be real-time (for performance reasons) and therefore a site like City Game Tracker, which allows you to see who is online at a particular time, will still be useful. Badge tracking sites typically provide quite a bit more functionality than simply listing badges - showing uncollected badges, for instance, or listing the top badge collectors on a server - so those will still be useful as well.

As for HeroStats, the main impact will be in the data we're sending to the community websites. Some, like City Game Tracker, probably won't change at all. Others, like the badge sites, might start getting their information from the City Vault and not need HeroStats at all. It's difficult to say until we see the City Vault in action, and have some idea how easy or difficult it will be to use that information.

Why does it take so long to load HeroStats?

I've seen a few people ask this question in the past. I freely admit that HeroStats takes a good long time to start up initially. And to make matters worse, while it's starting, it eats up most of your CPU so other programs are impacted. So why does this happen?

The answer, as in most programming, comes down to a trade-off. HeroStats makes heavy use of regular expressions (also known as regexs) to parse the text of the combat log into the stats that everyone wants. Regexs are a great way to recognize text messages and get information out of them, but they require a good deal of setup time if you want them to run quickly. This is known as compiling the regexs and it takes a lot of your computer's CPU time.

And we certainly want them to run quickly. HeroStats scans the CoX client chat tabs twice every second to look for new messages. Every time a new message is found, we have to run through the regexs to figure out what that message means. The code is pretty smart about checking the most common regexs first, but it can still take quite a while to go through them all, especially if your message happens to be one that isn't used very often.

So we want the regexs to run quickly, but in order to do that we need to spend the time to set them up first. And this is why your HeroStats takes a while to start up. It's loading up all those regexs so that while you're actually playing the game, your statistics/timers/etc. are being pulled quickly from those log messages.

Tracking Incoming Damage

Over on the CoH forums, there was an interesting question posted:

Can HeroStats separate out sources of damage?

Short answer: In theory, yes. In practice, no.

Long answer: Damage messages have a ton of different formats. Some show mob names, some show the power the mob used, some show damage type, and various combinations thereof (including none of the above).

Because of all this variety, it would take an inordinate amount of effort to track every single variation of every damage message. So I made a decision a while back that the only thing we would try to keep track of is damage type (and amount, of course). There's always a few messages with "Unknown" type, but most are tracked.

Even with this approach, we have dozens of different damage message formats to maintain, and every new patch seems to add a few more. It's really not feasible to keep up with all the various possibilities.

Windows Vista Compatibility

Ever since the Windows Vista release, I've gotten a lot of questions about using HeroStats with Vista. I haven't upgraded any of my own computers to Windows Vista, mainly because of hardware driver issues and because I really have no need for any of the new features - XP works fine for what I need. So my ability to develop for Vista is limited.

Some other folks in the community do use Vista, and they've provided some tips which can be found in our Known Issues. There are two main tips: run HeroStats in Windows XP compatibility mode, and use .NET Framework 2.0. Both of these things are effectively allowing you to run HeroStats on Vista as if it was running on XP.

Inside Buff and Recharge Timers

The timers in HeroStats are a source of many questions. Often the issue is missing timers, but there are lots of interesting things about timers beyond just making them show up. In this post I'll talk a bit about how the timers work. (By the way, if you're missing timers, the problem is almost always related to missing chat channels on your in-game setup. See the Known Issue in the Release Notes about this problem.)

HeroStats shows two types of timers - Buff timers and Recharge timers. Buff timers count down how long a particular buff (or in some cases debuff) will be effective. Probably the most commonly used buff timer is Hasten - it lasts for 120 seconds from the time you activate it. Recharge timers count down how long until a particular power is recharged. Just about every power will show a recharge timer, with the exceptions being powers that recharge instantly (i.e. Sprint).

All timers share some features that make it easier to keep track of them. When a timer drops below 30 seconds, it'll change color to yellow. Under 5 seconds, it'll turn red. This makes it easier to see at a glance which powers are about to recharge, or which buffs are about to run out.

Buff timers always start at the same value. No matter what else is happening to your character, the buff length is constant - Dull Pain will always last 120 seconds, for example. These timers are activated when a combat log message is received showing that the buff started. For instance,
when a Force Field character casts Deflection Shield on you, a message like this appears in your combat log:

{PlayerName} protects you from Smashing, Lethal and Melee attacks with a Deflection Shield and grants you some resistance to Toxic damage.
That message triggers HeroStats to start the buff timer. It will continue to count down until the buff expires. Assuming you don't suffer any major server lag (or zoning times), the buff timer should reach zero and disappear at the same time that the buff runs out.

There are several types of buff timers. Self buffs are powers that work on yourself (i.e. Hasten, Dull Pain). AoE buffs are powers that work on you and those around you (i.e. Recovery Aura, Group Invisibility). Target buffs are powers that work on whatever you have targeted (i.e. Speed Boost, Fortitude). The type of buff determines how the buffs are categorized in the HeroStats in-game window. For an AoE buff that you cast, you may see something like "{YourName} -> Group", while a Target buff that you cast will look more like "{YourName} -> {TargetName}".

Buff timers are also used for a few special cases. Inspirations can be shown as buffs, so that you can see how long you have before they wear off. You need to enable the "Show Inspirations as Buffs" setting in the Hero Configuration options (In-Game tab) for this to work. Also, some "pet" powers show as a self-buff, so you can see when those pets will expire. Examples of this include the Warshade Extracted Essence pet, Quicksand, Tar Patch, and similar powers. And finally, some environment effects are shown as buffs - for instance, the buffs and debuffs from exploding Lab Equipment on the tech-lab maps have timers.

Buff timers are configured by the HeroStats developers. I won't get into the details here, but if you're interested, the ChatMessageRegexs.dat file in your HeroStats directory contains some documentation on how it works.

Recharge timers show an estimated time until a power is recharged. Note the word "estimated". Because so many things can affect recharge - enhancements, outside buffs, enemy debuffs - there is no good way for HeroStats to know the exact time until the power will be recharged. It can, however, make a pretty good guess based on the last recharge time of that same power. That's the number you see in the recharge timer - the time it took for that power to recharge after the last time you used it.

How does HeroStats know that last recharge time? Each time you activate a power, the combat log contains a message. The activation message usually looks something like this:

You activated the {PowerName} power.

When a power recharges, another message is logged, looking something like this:

{PowerName} is recharged.

HeroStats keeps track of the time between these two messages - that's the recharge time of the power. Internally, HeroStats keeps a table of every power you've used in the current session and the recharge time for each. Whenever one of those activation messages is received, it looks up the power name in that table and displays a recharge timer.

Example: Let's say you use Hasten, and it takes 180 seconds (3 minutes) for it to recharge. The next time you use Hasten, HeroStats will display a 180 second recharge time. But then let's say you team with a Kinetic character and they Speed Boost you. Now Hasten recharges in 150 seconds. The current timer won't change - it'll still be counting down that 180 seconds. But the next time Hasten is used, the timer will be 150 seconds.

You may already have noticed the hole in this logic - what about the first time you use a power? There's no "last recharge time" on the first use - the power won't show up in that internal table I mentioned above. For this reason, HeroStats will not display a recharge timer for the first use of a power during a given play session. This is a commonly asked question, so I wanted to highlight it. You should see a recharge timer on your second use of any given power. This can be a bit confusing, especially on long-recharge powers like Unstoppable or Power Surge.

Is HeroStats accurate?

In my semi-regular browsing of the official forums for HeroStats questions, I came across a discussion about whether HeroStats tracking is really accurately recording what's actually going on in the game. I want to clear up some misconceptions that I've seen in some of the posts. First of all, the many people that have pointed out that the data recorded by HeroStats is not 100% accurate are absolutely correct. Leaving aside the philosophical discussion on whether any non-finite activity can be 100% accurate, I can point out that there are three main reasons for inaccuracies:

1. There are some powers that do not have combat log messages reflecting the actual behavior of the powers. I've documented these to the best of my knowledge on the Power Notes page of the HeroStats website. If anyone sees something incorrect or missing in that list, by all means report it.

2. The way HeroStats receives its information is passive - the scanning engine has read-only access to the information available to the game client, specifically the combat log. Because of this, HeroStats has no control over what is being done by the game client. This was a conscious design decision to avoid any possibility that HeroStats could directly influence gameplay (which is a EULA violation). What this means is that the game client could change the information being read in the middle of the scanner processing. When this happens, it's possible to miss information. I've got safeguards in place to minimize this, so it's rare, but it can happen and nothing short of violating the EULA by modifying the game client will prevent it. It's worth noting that this behavior is very dependent on the performance of your machine - fast machines with lots of RAM will see very few missed messages, while slower/older machines may see more.

3. We who work with HeroStats are only human. (Really.) We can and do make mistakes. Fortunately we have lots of people who are very willing to point these out to us, and whenever possible we fix the problems ASAP. In the cases where we can't fix something, we document it in either the Power Notes or Known Issues.

Having said all that - I believe HeroStats is a useful statistical tool, even without 100% accuracy. I can't give you any kind of numerical value representing how accurate it is for you, because that will vary based on the performance of your own particular setup (#2 above) and the combat log messages available for your powersets (#1 above). But I firmly believe that HeroStats captures the vast majority of information available in CoX correctly, and those tests that I've run on my own setup have confirmed that to my satisfaction. (Specifically, I tested my Fire tanker running Blazing Aura against several mobs and counted the ticks. This is the worst-case scenario for HeroStats, with multiple identical combat log messages being processed at once. Over 98% of the 2000+ ticks were properly recorded. In the vast majority of real play situations, the percentage of missed messages should be much lower, since we can do some error correction when the messages aren't identical.)

If you choose to believe that HeroStats isn't doing a good job of capturing statistics, that's your prerogative. But please, don't knock those that do use it unless you're willing to show some proof that it fails in a particular way. And by all means, report those issues that you do find, so we can correct them!