We have had a few reviews come in that are just one line. While it may be flattering
to read 'a must download, try it!' reviews like this are not
very helpful to the designer. This article will provide an updated
description of how to write quality reviews for Heaven Sites
that are both consistently scored between reviewers and helpful
to the creator and the potential downloader.
Angel SpineMan created the reviewing system for use on Age of
Empires Heaven, which has since been adopted for all our sites.
It has the 5-category system of Playability, Balance, Creativity,
Map Design, and Story/Instructions, as well as Additional Comments
at the end. It covers all the areas that a truly great scenario
should excel in. Each category is scored on a range from 1 to
5 with 5 being the best score and 1 being the worst. The total
score is the average of the category scores. The system is very
effective if it is used correctly.
At the beginning we had a small team led by an Angel to do all
reviewing. As such the reviews were very consistent since the
team wrote them all. That worked well for a while but as we
have opened up reviewing to the whole community, and our downloads
system allows posting by any member at any time, the reviews
have become inconsistent.
This inconsistency is troubling since scores can vary widely
depending on who is doing the review. That's not fair to the
designer or to the downloader. I know that there will always
be some variation in scores depending on the reviewer, but the
amount of variation is unacceptable and this document will detail
what is expected. This is to ensure that reviews remain a valuable
tool for both designers and downloaders.
General Guidelines
The first thing to always keep in mind is that reviews serve a double purpose.
They are written for both the scenario designer and for the
site visitor who is considering downloading the file. As such,
reviews need to praise the designer for things that are done
well and point out areas where the designer can improve. Reviews
also need to provide enough information about the scenario so
that potential downloaders will know if the scenario matches
their interests. Obviously, don't give away the plot or reveal
secrets that should be discovered while playing, but let the
downloader know what the scenario is about, if it's mainly fighting,
or mainly rpg (role player game) or mainly puzzles or a mix
of everything. Whatever the case, a downloader should know what
to expect from a scenario after reading a review.
Try as hard as possible to avoid vague statements in reviews.
Make sure that your review answers more questions than it raises.
Don't say something like "The first part of the 2nd scenario
was good" or "the part with the wolf could be improved" without
providing further explanation. Try to always include an example
from the scenario to back up any points that you make. If you
are pointing out something to the designer that you feel could
be improved, try to provide some ideas that the author could
build on. Do as much as you can to help the designer improve
his work.
All scenarios have good aspects and bad aspects. Try to always
say at least one good thing about any scenario you review and
never insult a designer even if you are handing
out a score of 1.0. Be honest but make every effort to encourage
the designer to do better next time.
Always spell check your reviews. I've seen far too many reviews that take points off a rating because of poor spelling yet the review itself is full of errors. Don't embarrass yourself - spell check your work.
Lastly, the review should contain a short explanation of why
you scored each category the way you did. This does not need
to be lengthy, sometimes a sentence is enough but other times
a paragraph for each category is necessary.
Granary Rating System
The Granary's rating system consists of 5 categories: Playability, Creativity,
Balance, Map Design and Story/Instructions. Each category is
rated by a reviewer on a scale of 1-5 and the overall score
is the average of the other 5 scores. A detailed explanation
of the system is below.
1 - Crash and burn. Indicates a lack of effort and/or bugs.
2 - Below average. It's not terrible but it still needs some work to be acceptable.
3 - Average. A decent job that is effective but no more work was done than necessary.
4 - Above average. An excellent job with some extra work done to make playing really enjoyable.
5 - Exceptional. A tough score to receive - A wonderful job with lots of extra work, precise crafting and thorough testing.
Before I detail what is expected for each category, there are some general scoring guidelines to make note of. All of the categories are subjective, some more than others, but try to be as consistent as you can with your own scoring. Also, take special note of a few things that should NOT affect the score of a scenario. These things should be noted in the review, but they should not affect the rating scores in any way.
The length of a scenario or how many scenarios are in a campaign
should make no difference. There is no rule that says a scenario
must last more than 15 minutes or that a campaign must include
at least 3 scenarios. The scores should only reflect how good
the scenario was while it was being played. A great 5 minutes
should score higher than a mediocre 2 hours. No reduction in
score should be made based on the length.
A scenario should not be penalized for not including special extras like music files or custom AI files. These extra items are great if used effectively and certainly can boost a score but a scenario should not be rated poorly just because of a lack of extras. A scenario should still be able to achieve a score of 5.0 even without using special extra files. The AoE design tool is rich with extras already that a designer should not be required to use custom files if they can achieve their design goals using what is already built into the design tool.
Playability
Playability is probably the most subjective element of the scoring. It is a gauge
of how much fun you had playing this particular scenario. Try
to only review scenarios that use a style you enjoy. For example,
if you hate playing RPG scenarios don't try to review one since
you probably didn't enjoy it and will likely give an unfair
review because it's not to your personal taste. If they are
not your kind of thing and you feel you must review them anyway,
keep it fair and don't mark them down for it.
There are no specific criteria for how a score is given in Playability,
but there are a few things that can have a negative effect.
Trigger bugs and scripting bugs (where the editor has triggers
or scripting), victory condition bugs and other such things
can ruin a scenario's playability. Lag is another playability
issue that a scenario can be marked down for, although if you
have a slow machine the fault may be with you, not the scenario.
If a player is ever confused about the next goal to accomplish,
that's a playability problem. If a player can complete an objective
in a way that the author obviously did not intend to be possible.
i.e. there's a hole in a wall that allows the player to skip
half the scenario, that's a playability problem. Beware, though,
as other shortcuts could be deliberate, a reward for those who
find them. I always build in an easy way and a hard way to do
any of my scenarios, so be absolutely sure it is a slip up before
you label it as one.
Consider also a scenario’s replayability value. Is it one you could play again and again, or maybe dig out at times for another go? Then it should score highly. Otherwise, anything that adversely affects your enjoyment of a scenario can be deducted from the Playability score.
Balance
Balance is also somewhat subjective since each player is of a different skill
level and what might be perfectly balanced for one player might
be too easy or too hard for another. As a reviewer you must
take your own skill level into account when giving a balance
score. A perfectly balanced scenario should provide a challenge
for a veteran player. Most people who are downloading scenarios
from the internet have at least played through the campaigns
included with the game and have good game knowledge.
Balance is slightly different from Playability and Creativity
although it involves elements of both. Balance represents the
difficulty and flow of the work. Was it beatable but not too
easy? If you had to build and upgrade your units between battles,
was this done well and fairly, not attacking you with too many
powerful units before you could fight them off? Are you still
challenged by the scenario even if you've already won it? If
the answer to all these is yes, it gets a high balance score.
Most well-balanced scenarios should not be able to be completed
without the player losing a few times. If a player is able to
complete the entire scenario the first time, the scenario is
probably too easy. On the other hand, a player should not need
to reload 15 times to get by a certain part of a scenario. That
is frustrating and the scenario is possibly too difficult. The
ideal scenario balance happens when a player gets stuck but
knows it is possible to complete. A player should not win by
luck; the scenario should be constructed so that a player can
learn from mistakes, try different approaches and use his skill
to complete the objective.
With great balance a scenario is still fun and challenging to
play even if you’ve already beaten it. This ties into playability
since it is usually one of the big reasons why it is replayable.
Creativity
This area is probably second in subjectivity behind playability. Creativity is
found in all aspects of a scenario - map tricks, map design,
the story, what units a player is given, objectives, AI, etc.
Every aspect of a scenario factors into creativity. One thing
to be careful for is not to knock points off of creativity if
the designer uses a trick you've seen used in another scenario.
There's nothing wrong with using the same trick that someone
else used and no reason to deduct points because of that. Furthermore,
if the trick is one you have seen before but used in a way you
haven’t, that is creative, therefore using known techniques
in new and interesting ways should actually increase the score.
Probably the biggest creativity factors are the starting position
and the victory conditions. For example, any scenario that starts
with a TC and three villagers with a conquest victory condition
and no other objectives to achieve than you would get in a regular
RM, is simply not very creative. The farther a player gets from
a random-style scenario, the better the creativity score, with
one exception. A scenario with well defined interesting objectives
but still containing enough random factors for it to be played
in many different ways is very creative and fun. It adds a lot
to replayability value as you try to beat it in different ways
and that deserves recognition.
Map Design
A rating of 1 is for a map with little or no real work put in, such as a map with
large blank areas with lots of square areas and straight lines.
These maps look completely unrealistic and are quite unattractive.
In a fantasy scenario, of course, the landscape may have a deliberately
unreal look to add to the atmosphere so it would be unfair to
criticise it for looking "unrealistic". However, we
are talking about cases where no effort has been made to make
the map interesting in any way at all.
A random map is fine if it works for the scenario but shouldn't
by itself get much more than a score of 2. If a random map has
been generated and then tweaked it will score higher as this
shows inventiveness and adaptation, both hallmarks of good design.
Ingo is an example of a great designer who often starts with
a random map then customises it.
A rating of 5 is for an outstanding map with lots of special
details and concentrated effort to make the map much better
than a mere random map.
More features on the landscape do not, however, make a better
scenario if they have been thrown on with no care given to layout.
Some maps have lots of features that have obviously been thrown
in because they are expected, to try and score highly, yet they
are messy, cluttered and may even hinder gameplay. Others are
inventive works of art that add interesting landmarks to the
map. Map design is about more than just terrain features and
tricks to create bridges and so forth, it is about how artfully
such things are applied.
Only the portion of the map that can be seen during play should
be scored. If there are empty/unimaginative areas that a player
never sees because they are outside the playable part of the
map, that should not affect the map design rating.
Finally, many people get hung up on what I call the "style
over substance" mentality. Lots of eye candy and a pretty
landscape but the game is dull as ditchwater to play. Even if
there isn’t much eye candy, one of the most important aspects
of map design is the effect it has on playability. Generals
in real battles have used terrain features to gain military
advantage, such as funnelling enemy troops into a tight space
for easier victory, putting archers on hills for extra damage
and so on. Some of the best maps I have ever played had little
eye candy but the way the land had been sculpted made for interesting
battles where a wide range of tactics could be used. Gordon
Farrell is a good example of a designer who never used much
eye candy but the map was carefully designed for gameplay and
tactics. To mark them down for lack of eye candy misses the
point.
So don’t just look at the pretty flowers – look at how the terrain
features enhance gameplay. Does the layout make for some interesting
battles and allow a range of tactics to be used? Then it scores
high. If the designer has only thought about how it looks and
not how it plays, no matter how pretty the map it should not
get a 5.
Story/Instructions
If there is no story or instructions, the score is easy. It's a 1. If there are
instructions but no story, you may decide it’s worth 2-3. Outstanding
instructions with a good story, good hints and history score
highest. If the instructions are wrong, misleading, confusing
or inadequate, the rating goes down.
Some other guidelines on scoring this category: An introductory
bitmap is a nice touch and a good image can often raise the
score, however, an introductory bitmap is not required to score
a 5. It helps, but it's not an absolute requirement. Hints and
History are not required either but they can also help boost
a scenario's score. While a bitmap, hints and history are not
required it would be difficult to give a rating of 5 if all
three areas are missing. The rating should not be affected based
on whether the story is fictional or historical. It doesn't
make a difference as long as there's a story that draws the
player into the scenario and the story itself is well told and
internally consistent.
The last item that factors into the rating of the story and
instructions is grammar and spelling. A designer should be diligent
in this area of his scenario since it's very easy to copy the
text into a word processor and spell check the instructions.
There's no excuse for having spelling errors in a scenario,
it simply shows a lack of effort on the part of the designer.
The only exception I make is for designers whose primary language
is not English, where I am a bit more lenient with them.
Additional Comments
This area is to be used to add any missed thoughts or points you would like to
cover. Always try to be positive when adding any comments. Constructive
criticism is always welcome as it aids the designer to improve
his/her creation or the following campaign. The last thing anyone
wants to see when reviewing their creation is “Don’t down load
this as it sucks or worse.” You will also get a warning or ban
for doing so. Be diplomatic and helpful at all times.
Final Thoughts
The above instructions are specifically for writing scenario and campaign reviews.
However, we are now allowing reviews of all file types available
for download so you can rate and write reviews on Mods, Random
Maps, AI Files, Utilities and any other files here! These reviews
will not use a 5 category system, instead you will give a single
rating to the file. When you write your review, simply include
enough information to explain why you gave the rating that you
did.
Make sure that you are familiar with all guidelines before writing
a review. If you do not adhere to these guidelines, you risk
having your review removed or converted to a comment. In the
event that your review is removed, there's no need to be discouraged.
This is not an insult or derogatory statement about you, simply
an indication that you need to go over what you wrote and see
what you can improve. No one is asking you to have the style
or technique of a professional critic; all that we ask is that
you take the time to write a decent review and think about what
you are writing.
Now that you've made it all the way through this document, you are ready to write reviews!
