Halo Modding Primer The Halo Editing Kit

Gearbox Software developed "the Halo Editing Kit", abbreviated as "HEK", for use with Halo Custom Edition, so that Halo gamers using a PC can design new multiplayer maps with the Halo Editing Kit and then run them with Halo Custom Edition. The Halo Editing Kit consists of three main programs: Tool, Guerilla, and Sapien. Tool is a console program which means that it is used for distinctly linear tasks and does not provide a user interface to display. Tool is used to generate intermediary files called "tags" and to compile tags into finished "map cache files", or "maps", which is the file format used by the Halo game to store playable game levels and related asset archives. Guerilla is used to modify asset-based tags that were converted by Tool, and can also be used to generate all the other tag files that do not contain assets converted by Tool. Sapien, which was notably developed over several years by Matthew Noguchi, is the HEK program used to view and develop a game level's scenario.

Installation

When you install Halo PC, you will need to use a Microsoft software product license key. The license key that comes with your copy of Halo PC is located on an orange sticker above the disc holder inside the disc case. The product key for Halo 2 Vista is also on an orange sticker above its disc holder inside the disc case. MacSoft's Halo PPC also comes with a CD Key which is located on the rear cover of the manual pamphlet included in the box with the installation disc. The format of the CD keys for Halo PC and Halo 2 Vista is capital letters and numbers of the sequence XXXXX-XXXXX-XXXXX-XXXXX-XXXXX, whereas the format for Halo PPC is capital letters and numbers of the sequence XXXX-XXXX-XXXX-XXXX. Product keys are meant to be privately used by the software to identify a player; each product key should only be used by one person's copy of the Halo software. They abstractly represent a proof of purchase and typically should not be shared. The installation software provides an installation ID, but this is typically ignored; the product key is more significant to the player. While MacSoft supported its Universal Binary version of Halo for Mac, they mentioned details about re-using Halo PPC license keys; Halo UB is a separate purchase from MacSoft that comes with its own license key. Halo Mini Demo, which is essentially a community-modified version of Halo UB, does not require a CD Key or a license key of its players.

Halo Custom Edition's installer requires that you provide a Halo PC license key. Therefore, both Halo PC and Halo CE are considered licensed software, and Halo CE requires that you have a Halo PC product key. However, it is possible to install the Halo Editing Kit without having installed Halo Custom Edition. But Halo Custom Edition is required to play the map files that you have compiled using the Halo Editing Kit. So in essence, the typically installation process is to first install Halo PC, then Halo Custom Edition, and finally the Halo Editing Kit. However, if you do not own a copy of Halo PC and do not have a valid product key for it, you can only install the Halo Editing Kit, and you will not be able to use Halo Custom Edition to play your new map files.

Directories

Halo PC's default installation directory is "C:\Program Files\Microsoft Games\Halo". Halo Custom Edition's default installation directory is "C:\Program Files\Microsoft Games\Halo Custom Edition". The Halo Editing Kit installs into the Halo Custom Edition program's folder. Note that Halo is a 32-bit program, and so the default installation directory on 64-bit computers running 64-bit versions of Windows would be "C:\Program Files (x86)\Microsoft Games\Halo" and "C:\Program Files (x86)\Microsoft Games\Halo Custom Edition". The 32-bit programs usually install into the "Program Files (x86)" directory instead of the "Program Files" directory when the Windows operating system is 64-bit. The Halo Editing Kit installs two working directories within the Halo CE directory: a folder called "tags" and a folder called "data". When you use the Tool program to convert data like images and sounds into tags like .bitmap and .sound, you place the data files into folders within the "data" folder, run the appropriate Tool command, and Tool generates the tags and places them within coinciding subdirectories within the "tags" folder. The HEK programs Tool, Guerilla and Sapien are designed to only operate with the "working tags directory". To avoid issues with programs that are hard-coded to use the default installation directories for the Halo games, it is recommended that you use the default installation directories when installing Halo, Halo Custom Edition, and the Halo Editing Kit.

Halo for PC and Mac also install saved game information and certain settings files within the Saved Games and Documents folders within the user's directory. These file paths are hard-coded into the game, so you do not get an option to customize their location. Therefore, if a user file is not in its default location, it will be considered non-existent.

Differences from Xbox

Halo for Xbox uses compression for its files. The Xbox version of the game also uses a different set of shader and model tag types from the Gearbox ports and the MacSoft ports. So for example, the appearance of teleporter entrances varies noticeably between the Xbox version of Halo and PC / Mac versions of Halo. The port of Halo between Xbox and PC / Mac is not exact. Gearbox Software included "optimizations" and MacSoft did its own overhaul with the Halo Universal Binary version. Halo PC and Halo Custom Edition also differ from each other.

The format and directory structure for Halo Xbox files is unique, since it uses compression and optimizations were seen as very necessary. The format of Halo PC files, notably the map files which contain level data, is practically the same as the format of files for Halo Full for Mac. Halo Demo and Halo Trial have about the same format of files as each other, different from the commercial versions Halo PC and Halo Full PPC, but the Halo Demo and Halo Trial programs have different quirks that can cause malformed or illogical map files to work on one but cause the other to crash. Halo Custom Edition uses a different format for its files, with its map files more heavily reliant upon common resource map references. The common resource map files for the games are bitmaps.map and sounds.map, where bitmaps.map contains common image data and sounds.map contains common audio data. Resource maps are used so that each scenario level stored in a playable .map file does not need to contain large swaths of duplicate data.

Maps

Playable levels are stored within .map files found within the "maps" folder in each Halo installation directory. (Halo PC uses "MAPS" and other capitalized folder names in the Halo installation directory, but using a case-insensitive computer setup means that "MAPS" is the same as "maps"; following the unix and Internet norm, lowercase is preferable and underscores are often necessary in lieu of spaces in subdirectories and file names. It is recommended to always use lowercase and underscores in lieu of spaces when using the Halo Editing Kit. Also note that unix uses the forward-slash "/" directory separator whereas Windows uses the back-slash "\" directory separator.) The .map files that contain playable information are of one general structure, different from the resource map files bitmaps.map and sounds.map. Halo CE and the HEK have a loc.map file which is used by Tool to help determine which tags are considered common; removing the loc.map file from your Halo CE's "maps" folder will result in Tool copying every tag that a compiled .map file references into the .map file during its compilation. Copying resource tags into a playable .map file is known as "internalizing" the tags. When a map has internalized tags, it potentially contains data in duplicate of the bitmaps.map and sounds.map resource map files -- the bitmaps or sounds from the standalone tags in the HEK tags directory are copied into the map file, even though their data could be already present in the resource maps bitmaps.map or sounds.map. When Tool does not find a loc.map, it will attempt to create a new loc.map, but not give it any data. So to switch back to normal map compilation and internalization functionality, the user would need to place their original loc.map file back into their Halo CE "maps" folder. Normal functionality only "internalizes" novel tag data into a map; only content that is not present within the two standard resource maps. Note that ui.map is also a playable level, but is unique in its use as the main menu for the game. Halo only loads one .map file at a time into memory, although it keeps it in memory once it is loaded. Halo PC and Halo Custom Edition initiate file locks on map files, and Halo Custom Edition only loads all the map files it can find in its "maps" folder all at once when it starts, so that while they are running, the .map file is not really modifiable. Halo Mac and Halo Demo do not lock modifications to the map files, and changes made to the file will be visible after a map file is reloaded, such as when choosing "new game" within the game menu while the map level is being played. Making map files work with Halo Demo or Halo Trial requires changing the format of the map file from its Xbox, commercial, or Custom Edition formats, and renaming the file as "bloodgulch.map" to replace the presence of the former "bloodgulch.map" file located in the game's "maps" folder. Halo Demo and Halo Trial look for the single "bloodgulch.map" as the multi-player level and the single "b30.map" as the single-player level. The commercial versions Halo PC and Halo PPC contain more information in their resource maps than the Halo Demo and Halo Trial versions. Resource maps are not normally interchangeable in equivalency between ports, just like the other .map files and user files are not interchangeable in equivalency between ports. The exception to this is that certain Halo PC and Halo Mac files, notably their .map files, are practically or exactly equivalent. This is also the case between Halo Demo and Halo Trial.

Tags

Playable maps contain game level information stored within tags. Essentially, playable map files consist of Halo Editing Kit tags that have been processed by Tool. When Tool processes tag files from the HEK "tags" directory, compiling them into a map cache file using the "build-cache-file" command, Tool adds additional information to the tags and removes the irrelevant file headers from them. Tool seeks out each tag in the tags directory that is referenced in chain starting with the two primary base tags, which are the globals tag located at "globals\globals.globals", and the scenario tag at hand, which is of a custom location within the tags directory which was specified as a parameter using the "build-cache-file" command. Tool organizes the tag data, sets up a tag index, and applies calculated file data internal offsets for each tag in lieu of the directory paths of the referenced tags which are stored as strings within the HEK format of the tags. Since the map file gets loaded directly into Halo's memory, Tool applies calculated offsets. Note that HEK tag files store data in Big Endian, and byte swapping occurs when Tool handles the map data, so that the Big Endian tag data is converted into Little Endian for Halo to read directly into its Little Endian memory space. This is why, for example, a scenario tag's referenced Binary Space Partition's tag class as "sbsp" for .scenario_structure_bsp HEK tags appears reversed as "psbs" in compiled .map files: the four bytes are reversed in order, from s-b-s-p to p-s-b-s, since the HEK tag file data is in Big Endian byte order and the map cache file's converted tag file data is in Little Endian byte order.

Tag Classes

Halo 1 has 83 tag types, or tag classes, that are recognized by the Halo Editing Kit programs. Tags come in different types. For example, vehicle tag types store information about vehicles in the game. Each tag of a certain type provides information for the game to use. When you add a tag to a playable map file, you add information of a specific type to that map, for example, a new kind of vehicle to exist in the game level. The Halo Editing Kit takes all the various tags that are somehow referenced through the main tags, as already mentioned, and compiles them into a playable .map file, and then you can play the level in Halo Custom Edition. Only 78 of the 83 tag classes are directly used with the HEK; the 5 others are somewhat abstract primary or secondary super-classes. Furthermore, not all 78 tag classes are implemented by the Halo game.

Here is a list of all the Halo 1 tag classes, presented as the four-letter tag class ID, the HEK tag file extension, and a name describing the role of the tag type:

  • actr / .actor / Actor
  • actv / .actor_variant / Actor Variant
  • ant! / .antenna / Antenna
  • antr / .model_animations / Animations
  • bipd / .biped / Biped
  • bitm / .bitmap / Bitmap
  • boom / .spheroid / Spheroid
  • cdmg / .continuous_damage_effect / Continuous Damage Effect
  • coll / .model_collision_geometry / Collision Model
  • colo / .color_table / Color Table
  • cont / .contrail / Contrail
  • ctrl / .device_control / Control Panel
  • deca / .decal / Decal
  • DeLa / .ui_widget_definition / UI Widget Definition
  • devc / .input_device_defaults / Input Device Defaults
  • devi / .device / Device
  • dobc / .detail_object_collection / Detail Object Collection
  • effe / .effect / Effect
  • elec / .lightning / Lightning
  • eqip / .equipment / Equipment
  • flag / .flag / Flag Cloth
  • fog / .fog / Fog
  • font / .font / Font
  • foot / .material_effects / Material Collision Effects
  • garb / .garbage / Garbage
  • glw! / .glow /Particle Glow Effect
  • grhi / .grenade_hud_interface / Grenade HUD Interface
  • hmt / .hud_message_text / HUD Messages
  • hudg / .hud_globals / General HUD Settings
  • hud# / .hud_number / HUD Numbers Settings
  • item / .item / Item
  • itmc / .item_collection / Respawning Objects Collection
  • jpt! / .damage_effect / Damage Effect
  • lens / .lens_flare / Lens Flare
  • lifi / .device_light_fixture / Light Fixture
  • ligh / .light / Light
  • lsnd / .sound_looping / Looping Sound
  • mach / .device_machine / Machine
  • matg / .globals / General Game Settings
  • metr / .meter / Meter
  • mgs2 / .light_volume / Volumetric Light
  • mod2 / .gbxmodel / Gearbox PC Model
  • mode / .model / Xbox Model
  • mply / .multiplayer_scenario_description / Multiplayer Descriptions
  • ngpr / .preferences_network_game / Multiplayer Character Settings
  • obje / .object / Object
  • part / .particle / Particle
  • pctl / .particle_system / Particle System
  • phys / .physics / Object Physics
  • plac / .placeholder / Object Placeholder
  • pphy / .point_physics / Particle Point Physics
  • proj / .projectile / Projectile
  • rain / .weather_particle_system / Weather
  • sbsp / .scenario_structure_bsp / Scenario Environment Structure
  • scen / .scenery / Scenery
  • scex / .shader_transparent_chicago_extended / Extended Transparent Shader
  • schi / .shader_transparent_chicago / Transparent Shader
  • scnr / .scenario / Scenario Environment Configuration
  • senv / .shader_environment / Environment Shader
  • sgla / .shader_transparent_glass / Transparent Glass Shader
  • shdr / .shader / Generic Shader
  • sky / .sky / Sky
  • smet / .shader_transparent_meter / Transparent Meter Shader
  • snd! / .sound / Sound
  • snde / .sound_environment / Ambient Effect
  • soso / .shader_model / Model Shader
  • sotr / .shader_transparent_generic / Generic Transparent Shader
  • Soul / .ui_widget_collection / UI Widget Collection
  • spla / .shader_transparent_plasma / Transparent Plasma Shader
  • ssce / .sound_scenery / Sound-Emitting Scenery
  • str# / .string_list / String References
  • swat / .shader_transparent_water / Transparent Water Shader
  • tagc / .tag_collection / Tag Collection
  • trak / .camera_track / Camera Track
  • udlg / .dialogue / Unit Dialogue
  • unhi / .unit_hud_interface / Unit HUD Interface
  • unit / .unit / Unit
  • ustr / .unicode_string_list / Unicode String References
  • vcky / .virtual_keyboard / Virtual Keyboard
  • vehi / .vehicle / Vehicle
  • weap / .weapon / Weapon
  • wind / .wind / Wind
  • wphi / .weapon_hud_interface / Weapon HUD Interface
Tag Class Triplets

The .device, .item, .object, .shader and .unit tags are the five abstract tag super-classes that are primary or secondary of these 83 mentioned tag classes; the actual usable tag classes are only 78 and are tertiary tag classes. Primary, secondary, and tertiary refer to the hierarchy of a tag class as it is shown within a map cache file's tags index. Primary and secondary tag super-classes describe the general categories of certain tertiary tag classes. Primary means the most general categorization and secondary is a more specific categorization, both describing the tertiary specific tag class. For example, both vehicles of the vehi tag class and bipeds of the bipd tag class are of the primary obje object tag class and the secondary unit tag class. So this essentially means that the game engine interprets these two specific tertiary tag classes as objects, and more specifically, units, and they both include object and unit sections of information within their details. As another example, weapons are objects, but instead of being units, they are items. So the primary abstract tag class for a weap tag is obje and the secondary abstract tag class is item. When a tag only contains one abstract class, it is listed as the secondary class, since there is no more-generic primary tag class applicable; so for example, a soso shader model is a tertiary class with a single abstract class of shdr shader, and there is no other abstract class for soso, and so shdr is the secondary abstract super-class for soso shader model tag types. Obje is not always a primary tag class: for example, with scen scenery, it is the only abstract class, and so is listed as a secondary tag super-class for scenery tag types. It helps to visualize it in this way: 3 columns, numbered from left to right as 3 (tertiary), 2 (secondary) and 1 (primary), the first containing the actual tag class in question and the other two columns containing optional secondary and primary tag super-classes by which the tag class in question can be categorized. For example:

  • tertiary main class / secondary abstract class / primary abstract class
  • bitm / - / -
  • matg / - / -
  • soso / shdr / - (soso Shader Model is a kind of shdr Generic Shader)
  • scen / obje / - (scen Scenery, such as rocks and trees, is a kind of obje Object)
  • bipd / unit / obje (bipd Biped, such as the player character, is a kind of unit Unit, and also more generically a kind of obje Object)

Here is an actual 832-byte segment from the tags index within the a10.map playable map file:

  • mtibˇˇˇˇˇˇˇˇ∫.‚óÓE@‡:X@
  • ososrdhsˇˇˇˇª/‚«ÓE@º;X@
  • mtibˇˇˇˇˇˇˇˇº0‚ÒÓE@t=X@
  • ososrdhsˇˇˇˇΩ1‚ÔE@P>X@
  • ososrdhsˇˇˇˇæ2‚NÔE@@X@
  • ososrdhsˇˇˇˇø3‚vÔE@¿AX@
  • ihcsrdhsˇˇˇˇ¿4‚£ÔE@xCX@
  • mtibˇˇˇˇˇˇˇˇ¡5‚ÕÔE@úEX@
  • rtnaˇˇˇˇˇˇˇˇ¬6‚˜ÔE@xFX@
  • hgilˇˇˇˇˇˇˇˇ√7‚E@`LX@
  • snelˇˇˇˇˇˇˇˇƒ8‚;E@¿MX@
  • mtibˇˇˇˇˇˇˇˇ≈9‚hE@0OX@
  • necsejboˇˇˇˇ∆:‚ùE@\QX@
  • 2domˇˇˇˇˇˇˇˇ«;‚ E@¿SX@
  • ososrdhsˇˇˇˇ»<‚˝E@¿WX@
  • mtibˇˇˇˇˇˇˇˇ…=‚3ÒE@xYX@
  • mtibˇˇˇˇˇˇˇˇ >‚ÉÒE@ÑZX@
  • mtibˇˇˇˇˇˇˇˇÀ?‚ŸÒE@ê[X@
  • ihcsrdhsˇˇˇˇÃ@‚ÚE@l\X@
  • mtibˇˇˇˇˇˇˇˇÕA‚7ÚE@¥]X@
  • hgilˇˇˇˇˇˇˇˇŒB‚XÚE@ê^X@
  • snelˇˇˇˇˇˇˇˇœC‚ÚE@_X@
  • dpibtinuejbo–D‚®ÚE@`aX@
  • 2domˇˇˇˇˇˇˇˇ—E‚√ÚE@LjX@
  • ososrdhsˇˇˇˇ“F‚fiÚE@.äX@
  • mtibˇˇˇˇˇˇˇˇ”G‚ÛE@ÙãX@

You can see how the four-letter tag class triplets of each indexed tag, presented here as one entry per line, shows the tag classes reversed, having been byte-swapped from the Big Endian HEK tag files to the Little Endian map cache file format suitable for reading into game memory. The first four bytes are the tertiary column, the second are the secondary column and the third are the primary column. The first entry, "mtib" is a bitm Bitmap tag entry which contains no super-classes. The second entry is "ososrdhs", which is soso Shader Model of the super-class shdr Generic Shader and no other abstract class, so there is nothing listed for the primary abstract tag class. Moving down, you can see that "necsejbo" means that scen Scenery tag classes have a single abstract class, therefore given as the secondary super-class, which is obje Object. Towards the end of the example, you see "dpibtinuejbo" or obje - unit - bipd, where the primary abstract class is obje Object, the secondary, more specific abstract class is unit Unit, and the actual tag class, the tertiary class, is bipd Biped.

13 tag classes have obje (object) abstract super-classes. They are listed here according to their internal indices:

  • -1 / obje / .object / Object
  • 0 / bipd / .biped / Biped
  • 1 / vehi / .vehicle / Vehicle
  • 2 / weap / .weapon / Weapon
  • 3 / eqip / .equipment / Equipment
  • 4 / garb / .garbage / Garbage
  • 5 / proj / .projectile / Projectile
  • 6 / scen / .scenery / Scenery
  • 7 / mach / .device_machine / Machine
  • 8 / ctrl / .device_control / Control Panel
  • 9 / lifi / .device_light_fixture / Light Fixture
  • 10 / plac / .placeholder / Object Placeholder
  • 11 / ssce / .sound_scenery / Sound-Emitting Scenery

10 tag classes have shdr (shader) abstract super-classes. They are listed here according to their internal indices:

  • -1 / shdr / .shader / Generic Shader
  • 3 / senv / .shader_environment / Environment Shader
  • 4 / soso / .shader_model / Model Shader
  • 5 / sotr / .shader_transparent_generic / Generic Transparent Shader
  • 6 / schi / .shader_transparent_chicago / Transparent Shader
  • 7 / scex / .shader_transparent_chicago_extended / Extended Transparent Shader
  • 8 / swat / .shader_transparent_water / Transparent Water Shader
  • 9 / sgla / .shader_transparent_glass / Transparent Glass Shader
  • 10 / smet / .shader_transparent_meter / Transparent Meter Shader
  • 11 / spla / .shader_transparent_plasma / Transparent Plasma Shader

There are 4 types of devi (device) tags, but they are not apparently indexed. They are listed here:

  • devi / .device / Device
  • ctrl / .device_control / Control Panel
  • lifi / .device_light_fixture / Light Fixture
  • mach / .device_machine / Machine

There are 4 types of item (item) tags, but they are not apparently indexed. They are listed here:

  • item / .item / Item
  • eqip / .equipment / Equipment
  • garb / .garbage / Garbage
  • weap / .weapon / Weapon

There are 3 types of unit (unit) tags, but they are not apparently indexed. They are listed here:

  • unit / .unit / Unit
  • bipd / .biped / Biped
  • vehi / .vehicle / Vehicle