How do you test a new set of rules?

A really rather useful  post over on Jonathan Freitag's blog 'Palouse Wargaming Journal'  >>Link<<  reviewing a new set of ACW rules from Helion got me thinking about the review process and what it is that is actually tested.  Over the years I have seen (and played) a number of rule sets that had good mechanisms but which were not a good reflection of the period of warfare they purported to cover.  Equally I have seen rules where there is a good game with good period flavour in there somewhere but the rules fail to properly bring it out.  Lastly there are those rules where everything is covered but they bog down under the weight of that detail (I have probably been guilty of homebrew rules like this).  Personally I believe that rules reviews should tell me what the reviewer would like from a set of rules and then tell me why the rules under review do or don't achieve that .  That's because I'm always going to place a higher value upon a review which is looking for the same things I am in a set of rules.

One thing that really puts me off any new set of rules is a lack of internal consistency.  For example defining the size of a standard unit and then using units representing an entirely different size in scenarios included within the rule book.  Next worst is not properly explaining rules, again something I really worry that I might be guilty of.  After a while drafting and redrafting there comes a point where you know what a rule is meant to do and somehow you simply cannot see that the rule as you have written it doesn't actually convey the meaning you have in your mind's eye.  Which I suppose is where good proof readers and editors come into their own.  One of my personal peeves with Helion is the poor proof reading and editing.  

For me the acid test is can I replay a historical battle with the rules and have (in no particular order):

- orders of battle as defined under the rules that properly represent the troops that were actually engaged on that day,

- decision points that reflect the key issues that the actual commanders had to deal with at the command level the rules focus upon,

- restrictions built into the rules that mirror the key restraints faced by the commanders in the period,

- did it have the feel of a battle of the period,

- was it fun or at least interesting to play (AKA the will I play these rules again test),

- the turn duration - weapon range- movement distance equation hang together properly (and be properly balanced).

I know that other gamers will have different priorities and some are not interested in historical gaming at all, but even in High Fantasy or Science Fiction rules their should be internal consistency not just in rules hanging together but in the trade off between cost/benefit and impact of the technology in those fictional worlds, I like my fiction to have some grounding in reality.

I can live a set of rules where I have to 'tinker' around the edges but not if the fundamentals are flawed.  I also like a set of designer's notes so I can see the process for choosing mechanisms and why aspects were included within or excluded from the rules.  I accept that a set of rules will not and should not include everything that could happen in the real world but it is nice to see the thinking that has gone into the choices made.