Heaven’s Downloads

granary.gif (4630 bytes)

Review System Description

Introduction

We have had a few reviews come in that are just one line. While it may be flattering to read 'a must download, try it!' reviews like this are not very helpful to the designer. This article will provide an updated description of how to write quality reviews for Heaven Sites that are both consistently scored between reviewers and helpful to the creator and the potential downloader.

Angel SpineMan created the reviewing system for use on Age of Empires Heaven, which has since been adopted for all our sites. It has the 5-category system of Playability, Balance, Creativity, Map Design, and Story/Instructions, as well as Additional Comments at the end. It covers all the areas that a truly great scenario should excel in. Each category is scored on a range from 1 to 5 with 5 being the best score and 1 being the worst. The total score is the average of the category scores. The system is very effective if it is used correctly.

At the beginning we had a small team led by an Angel to do all reviewing. As such the reviews were very consistent since the team wrote them all. That worked well for a while but as we have opened up reviewing to the whole community, and our downloads system allows posting by any member at any time, the reviews have become inconsistent.

This inconsistency is troubling since scores can vary widely depending on who is doing the review. That's not fair to the designer or to the downloader. I know that there will always be some variation in scores depending on the reviewer, but the amount of variation is unacceptable and this document will detail what is expected. This is to ensure that reviews remain a valuable tool for both designers and downloaders.

General Guidelines

The first thing to always keep in mind is that reviews serve a double purpose. They are written for both the scenario designer and for the site visitor who is considering downloading the file. As such, reviews need to praise the designer for things that are done well and point out areas where the designer can improve. Reviews also need to provide enough information about the scenario so that potential downloaders will know if the scenario matches their interests. Obviously, don't give away the plot or reveal secrets that should be discovered while playing, but let the downloader know what the scenario is about, if it's mainly fighting, or mainly rpg (role player game) or mainly puzzles or a mix of everything. Whatever the case, a downloader should know what to expect from a scenario after reading a review.

Try as hard as possible to avoid vague statements in reviews. Make sure that your review answers more questions than it raises. Don't say something like "The first part of the 2nd scenario was good" or "the part with the wolf could be improved" without providing further explanation. Try to always include an example from the scenario to back up any points that you make. If you are pointing out something to the designer that you feel could be improved, try to provide some ideas that the author could build on. Do as much as you can to help the designer improve his work.

All scenarios have good aspects and bad aspects. Try to always say at least one good thing about any scenario you review and never insult a designer even if you are handing out a score of 1.0. Be honest but make every effort to encourage the designer to do better next time.

Always spell check your reviews. I've seen far too many reviews that take points off a rating because of poor spelling yet the review itself is full of errors. Don't embarrass yourself - spell check your work.

Lastly, the review should contain a short explanation of why you scored each category the way you did. This does not need to be lengthy, sometimes a sentence is enough but other times a paragraph for each category is necessary.

Granary Rating System

The Granary's rating system consists of 5 categories: Playability, Creativity, Balance, Map Design and Story/Instructions. Each category is rated by a reviewer on a scale of 1-5 and the overall score is the average of the other 5 scores. A detailed explanation of the system is below.

1 - Crash and burn. Indicates a lack of effort and/or bugs.

2 - Below average. It's not terrible but it still needs some work to be acceptable.

3 - Average. A decent job that is effective but no more work was done than necessary.

4 - Above average. An excellent job with some extra work done to make playing really enjoyable.

5 - Exceptional. A tough score to receive - A wonderful job with lots of extra work, precise crafting and thorough testing.

Before I detail what is expected for each category, there are some general scoring guidelines to make note of. All of the categories are subjective, some more than others, but try to be as consistent as you can with your own scoring. Also, take special note of a few things that should NOT affect the score of a scenario. These things should be noted in the review, but they should not affect the rating scores in any way.

The length of a scenario or how many scenarios are in a campaign should make no difference. There is no rule that says a scenario must last more than 15 minutes or that a campaign must include at least 3 scenarios. The scores should only reflect how good the scenario was while it was being played. A great 5 minutes should score higher than a mediocre 2 hours. No reduction in score should be made based on the length.

A scenario should not be penalized for not including special extras like music files or custom AI files. These extra items are great if used effectively and certainly can boost a score but a scenario should not be rated poorly just because of a lack of extras. A scenario should still be able to achieve a score of 5.0 even without using special extra files. The AoE design tool is rich with extras already that a designer should not be required to use custom files if they can achieve their design goals using what is already built into the design tool.

Playability

Playability is probably the most subjective element of the scoring. It is a gauge of how much fun you had playing this particular scenario. Try to only review scenarios that use a style you enjoy. For example, if you hate playing RPG scenarios don't try to review one since you probably didn't enjoy it and will likely give an unfair review because it's not to your personal taste. If they are not your kind of thing and you feel you must review them anyway, keep it fair and don't mark them down for it.

There are no specific criteria for how a score is given in Playability, but there are a few things that can have a negative effect. Trigger bugs and scripting bugs (where the editor has triggers or scripting), victory condition bugs and other such things can ruin a scenario's playability. Lag is another playability issue that a scenario can be marked down for, although if you have a slow machine the fault may be with you, not the scenario. If a player is ever confused about the next goal to accomplish, that's a playability problem. If a player can complete an objective in a way that the author obviously did not intend to be possible. i.e. there's a hole in a wall that allows the player to skip half the scenario, that's a playability problem. Beware, though, as other shortcuts could be deliberate, a reward for those who find them. I always build in an easy way and a hard way to do any of my scenarios, so be absolutely sure it is a slip up before you label it as one.

Consider also a scenarioís replayability value. Is it one you could play again and again, or maybe dig out at times for another go? Then it should score highly. Otherwise, anything that adversely affects your enjoyment of a scenario can be deducted from the Playability score.

Balance

Balance is also somewhat subjective since each player is of a different skill level and what might be perfectly balanced for one player might be too easy or too hard for another. As a reviewer you must take your own skill level into account when giving a balance score. A perfectly balanced scenario should provide a challenge for a veteran player. Most people who are downloading scenarios from the internet have at least played through the campaigns included with the game and have good game knowledge.

Balance is slightly different from Playability and Creativity although it involves elements of both. Balance represents the difficulty and flow of the work. Was it beatable but not too easy? If you had to build and upgrade your units between battles, was this done well and fairly, not attacking you with too many powerful units before you could fight them off? Are you still challenged by the scenario even if you've already won it? If the answer to all these is yes, it gets a high balance score.

Most well-balanced scenarios should not be able to be completed without the player losing a few times. If a player is able to complete the entire scenario the first time, the scenario is probably too easy. On the other hand, a player should not need to reload 15 times to get by a certain part of a scenario. That is frustrating and the scenario is possibly too difficult. The ideal scenario balance happens when a player gets stuck but knows it is possible to complete. A player should not win by luck; the scenario should be constructed so that a player can learn from mistakes, try different approaches and use his skill to complete the objective.

With great balance a scenario is still fun and challenging to play even if youíve already beaten it. This ties into playability since it is usually one of the big reasons why it is replayable.

Creativity

This area is probably second in subjectivity behind playability. Creativity is found in all aspects of a scenario - map tricks, map design, the story, what units a player is given, objectives, AI, etc. Every aspect of a scenario factors into creativity. One thing to be careful for is not to knock points off of creativity if the designer uses a trick you've seen used in another scenario. There's nothing wrong with using the same trick that someone else used and no reason to deduct points because of that. Furthermore, if the trick is one you have seen before but used in a way you havenít, that is creative, therefore using known techniques in new and interesting ways should actually increase the score.

Probably the biggest creativity factors are the starting position and the victory conditions. For example, any scenario that starts with a TC and three villagers with a conquest victory condition and no other objectives to achieve than you would get in a regular RM, is simply not very creative. The farther a player gets from a random-style scenario, the better the creativity score, with one exception. A scenario with well defined interesting objectives but still containing enough random factors for it to be played in many different ways is very creative and fun. It adds a lot to replayability value as you try to beat it in different ways and that deserves recognition.

Map Design

A rating of 1 is for a map with little or no real work put in, such as a map with large blank areas with lots of square areas and straight lines. These maps look completely unrealistic and are quite unattractive. In a fantasy scenario, of course, the landscape may have a deliberately unreal look to add to the atmosphere so it would be unfair to criticise it for looking "unrealistic". However, we are talking about cases where no effort has been made to make the map interesting in any way at all.

A random map is fine if it works for the scenario but shouldn't by itself get much more than a score of 2. If a random map has been generated and then tweaked it will score higher as this shows inventiveness and adaptation, both hallmarks of good design. Ingo is an example of a great designer who often starts with a random map then customises it.

A rating of 5 is for an outstanding map with lots of special details and concentrated effort to make the map much better than a mere random map.

More features on the landscape do not, however, make a better scenario if they have been thrown on with no care given to layout. Some maps have lots of features that have obviously been thrown in because they are expected, to try and score highly, yet they are messy, cluttered and may even hinder gameplay. Others are inventive works of art that add interesting landmarks to the map. Map design is about more than just terrain features and tricks to create bridges and so forth, it is about how artfully such things are applied.

Only the portion of the map that can be seen during play should be scored. If there are empty/unimaginative areas that a player never sees because they are outside the playable part of the map, that should not affect the map design rating.

Finally, many people get hung up on what I call the "style over substance" mentality. Lots of eye candy and a pretty landscape but the game is dull as ditchwater to play. Even if there isnít much eye candy, one of the most important aspects of map design is the effect it has on playability. Generals in real battles have used terrain features to gain military advantage, such as funnelling enemy troops into a tight space for easier victory, putting archers on hills for extra damage and so on. Some of the best maps I have ever played had little eye candy but the way the land had been sculpted made for interesting battles where a wide range of tactics could be used. Gordon Farrell is a good example of a designer who never used much eye candy but the map was carefully designed for gameplay and tactics. To mark them down for lack of eye candy misses the point.

So donít just look at the pretty flowers Ė look at how the terrain features enhance gameplay. Does the layout make for some interesting battles and allow a range of tactics to be used? Then it scores high. If the designer has only thought about how it looks and not how it plays, no matter how pretty the map it should not get a 5.

Story/Instructions

If there is no story or instructions, the score is easy. It's a 1. If there are instructions but no story, you may decide itís worth 2-3. Outstanding instructions with a good story, good hints and history score highest. If the instructions are wrong, misleading, confusing or inadequate, the rating goes down.

Some other guidelines on scoring this category: An introductory bitmap is a nice touch and a good image can often raise the score, however, an introductory bitmap is not required to score a 5. It helps, but it's not an absolute requirement. Hints and History are not required either but they can also help boost a scenario's score. While a bitmap, hints and history are not required it would be difficult to give a rating of 5 if all three areas are missing. The rating should not be affected based on whether the story is fictional or historical. It doesn't make a difference as long as there's a story that draws the player into the scenario and the story itself is well told and internally consistent.

The last item that factors into the rating of the story and instructions is grammar and spelling. A designer should be diligent in this area of his scenario since it's very easy to copy the text into a word processor and spell check the instructions. There's no excuse for having spelling errors in a scenario, it simply shows a lack of effort on the part of the designer. The only exception I make is for designers whose primary language is not English, where I am a bit more lenient with them.

Additional Comments

This area is to be used to add any missed thoughts or points you would like to cover. Always try to be positive when adding any comments. Constructive criticism is always welcome as it aids the designer to improve his/her creation or the following campaign. The last thing anyone wants to see when reviewing their creation is ďDonít down load this as it sucks or worse.Ē You will also get a warning or ban for doing so. Be diplomatic and helpful at all times.

Final Thoughts

The above instructions are specifically for writing scenario and campaign reviews. However, we are now allowing reviews of all file types available for download so you can rate and write reviews on Mods, Random Maps, AI Files, Utilities and any other files here! These reviews will not use a 5 category system, instead you will give a single rating to the file. When you write your review, simply include enough information to explain why you gave the rating that you did.

Make sure that you are familiar with all guidelines before writing a review. If you do not adhere to these guidelines, you risk having your review removed or converted to a comment. In the event that your review is removed, there's no need to be discouraged. This is not an insult or derogatory statement about you, simply an indication that you need to go over what you wrote and see what you can improve. No one is asking you to have the style or technique of a professional critic; all that we ask is that you take the time to write a decent review and think about what you are writing.

Now that you've made it all the way through this document, you are ready to write reviews!

Go To Downloads

Back to Top