1. Principles
    1. How does it work?
    2. Terminology
    3. Types
      1. References
      2. Keystructs
    4. Hierarchy
      1. Naming
      2. Binary order
      3. Text order
      4. Specialized keystructs
    5. Internationalization
    6. Text and Bin
    7. References & Resolution
    8. Simple form
    9. Easy to write
    10. Structure and sources

Principles

RPL or Resource Replacement Language is a descriptive language meant to describe complex, generally packed data structures. It is meant to ascribe computationally useful meaning to this data so that it can be, for instance, translated into a presentable format.

If you have ever reverse engineered a ROM or other complex multimedia object, you may have taken notes to aid you in doing so. For instance, that the current font is stored at a certain position and contains an alphabet in a certain order, perhaps that the text using this font is located in this area or perhaps you've uncovered the whole table file. Imagine those notes working for you to export the text and image in a readable form: as JSON or some sort of script file and a PNG. Imagine being able to then edit that PNG to have the characters for your language or simply being able to change the text as it sits there and then reimporting your changes just like that. If you take your notes in RPL, that's exactly what it intends to be.

RPL does more than this, though, as it allows you to programmatically work with the data if you wanted to. You can use it to read in data from your own or other people's binary formats and work with it in an object-oriented way. For instance, if one were to make something like PokeMMO which uses the resources of official Pokemon ROMs as its own, RPL might be a good fit for such a project. It would allow you to verify that the user is using the correct ROM (or, of course, select from RPLs based on the ROM) and load its resources in a unified way. Once the proper RPL is selected, the client never has to know where the resources came from.

There are, of course, more uses than ROMs. For instance, one could use this system to manage custom archive formats for MMOs and other games. One could even use it in their own game for such a feat.

Many of you are probably familiar with C structures, so what's the difference here? You can fread and fwrite packed structures in C after all. The main difference is one of complexity. For example, C doesn't allow you to read in dynamically sized structures with one command where RPL does. RPL could do this with either an explicit length or even by just ceasing to read at a null terminator. RPL can link data across sections, such as the size of an archive's payload section being the sum of its contents' sizes. More importantly it can do much of this in very simple and sometimes transparent ways. C structures would also only cover the un/serializing and not exporting the files in standard formats which RPL makes a cakewalk.

So why not 010 Editor's Binary Templates? These are actually quite similar and I only have a surface-level familiarity with them. Functionally they are very similar though there seems to be a difference in goals: RPL is primarily focused on transport of data (ie. exporting to usable formats and importing back into the binary) while Binary Templates are focused on editing data. (TODO: Deeper investigation) 010 Editor seems to rely more heavily on programming to do certain things, like to handle text encodings, whereas RPL favors a purely descriptive style. One thing Binary Templates do better is conditional formatting, where you can add inline if statements to conditionalize structuring on, for example, format version. RPL cannot do this in as simple of a way and would require another depth to the object access for each conditionalized section (in terms of reading/using the data) and a more complex format selection system that is too detailed for this section (in terms of constructing the RPL).

So why not use existing tools? There are many existing tools for simple binary modification, graphic browsing, format conversion, text searching & replacing, etc. My trouble with this is that they generally have poor configurability so I cannot use them for my specific project. Though even when I can this can be a complex multistep process. In able to actually import the new data, especially if it was a custom format for a certain game, I would end up writing a custom script to handle it which were generally brittle and were only for importing.

Though there are existing all-in-one tools for ROM hacking. Most of the projects are dead but certainly the tools are still useful? At least, why did I design RPL instead of extending one of these applications? There have been two tools with a similar goal that I know of: ROMulan and The Console Tool. ROMulan is dead but it provided a console or BASIC-like scripting language that allowed you to extract and reimport binary data and automatically allocate new space for resources larger than the originals. Low Lines's TCT is stale as of 2012 but received a lot of support. It features a GUI that allows browsing certain types of ROMs as archives and viewing and editing their contents including sprites, models, and so forth. It can also function as a hex editor for other cases and features plugin support (I think?) for enhancing its abilities. It's a very powerful and well-built tool but sadly is not open source, making it difficult to extend. There are plenty of other tools that do one thing really well like Atlas or Tinke as well which are great but a lot of these types of tools don't support smaller consoles like the Pokemon mini, newer Tamagotchis, Adventure Vision, and so forth. I wanted something that would let me dive into a project with an unsupported console's game just as well as supported ones. Surely, consoles with their own libraries will be easier but I wanted it to still be relatively simple without having a library too.

So finally, why not just write your own tool? Rolling your own script generally produces programs that are not robust, unhelpful when they fail, possibly only manage one transport (import → bin or bin → export, rather than both), and so forth. If you want to write methods to serialize, unserialize, export, and import this can be up to 4x the code required for one thing. Even at its most minimal amount of effort, this is a C struct that can be directly serialized and unserialized, library calls to export and import, and then the code that converts to and from what the C struct contains and what the library expects. Generally speaking this is a lot of work and it grows exponentially. With RPL through Imperial Exchange, however, you may do all of this - un/serialize, export, and import - with only writing one struct.

How does it work?

The basic transport system works like this:

Unserializing      Export
        ╲         ╱
     Bin → Susp- → Expo
     ary ← ended ← rted
        ╱   │ ↑   ╲
Serializing │ │   Import
            ↓ │
        Application

The process of moving the binary data into usable, language-specific object structures is called unserialization. The reverse of this is, of course, serialization. These are essentially the fread and fwrite, respectively, to and from a C struct. Those language-specific object structures are what the diagram refers to as the data's suspended state. In this state an application using the RPL library can interact with the data. Finally, outputting the data to the file system as an individual, usably formatted file is called exporting. The reverse of this is importing. Any application may use the RPL library in any fashion, independent of the other transport options. It could import, do modifications, and export back out. It could unserialize, do modifications, and serialize the data back in directly. It could also generate data internally and simply serialize it to a binary or export it to a useful format.

One structure can handle all that? Yes! For example, here's the tile format for the Pokemon mini using only the generic graphics structure:

graphic SomeTile {
    base: $003d44
    # Format:
    dimensions: [8, 8]
    pixel: 0bw
    read: DULR
}

This describes a single graphic in the ROM at address 0x003d44. It is 8 by 8 pixels where each pixel is one bit each such that 0 is white and 1 is black. It writes (on export) and reads (on import) the pixels in this reading direction (Down → Up, Left → Right) such that the first pixel read (the MSB of the byte located at 0x003d44) represents the lower-left pixel in the final image:

^\  ^\  ^
| \ | \ | ...
|  `|  `|

The file would be exported to and imported from the current working directory as SomeTile.png. Thus in this small form you have just described everything you need to un/serialize, export, and import the image in question. This is hopefully a clean, pretty, readable, and notational form! Though RPL strives to be easy to write rather than easy to read, this should be readable if you're familiar with the struct type in question, graphic. If you aren't it should be trivial to look up at the command line.

In the vein of being easy to write, RPL doesn't want you copypasting things like formats! For multiple tiles you can do it like this, instead:

static {
    # Format
    dimensions: [8, 8]
    pixel: 0bw
    read: DULR
    graphic SomeTile    { base: $003d44 }
    graphic AnotherTile { base: $003d4c }
}

Which will export two tiles to the CWD, SomeTile.png and AnotherTile.png, from different locations in the ROM using the same format.

Linking data between sections of a RPL file is generally done with some sort of reference (see reference syntax for more details) which allows you to reference a named struct and a key within it. However, note that everything is structs, even values. The only hard distinction between the two is that values must have at least one data form (number, string, or list). Because of this, you can even reference properties of a value, like its serialized size (take a look at the value specifications for more information). The difference in referencing this is merely a difference of @Struct.key vs. @Struct.key.size as the former will actually generally assume the data (the only other option being a pointer but the difference is both meaningless and clear in context).

Terminology

There are multiple concepts of data representation that are distinct within each type and within the system otherwise. This section explains all of those concepts and terms.

Abstract form
From a Python perspective, these are the class instances which represent instantiated structures. It is one piece of the suspended data.
Author
The writer of a RPL file.
User
A person who is using a RPL file with some tool. Often, a user who did not write that RPL file.
Define
This is what's written as the value of a key in the RPL file or in the basic constructor (parenthetical or list form, not the struct-like form) of a keystruct. This may be different from what the data returned is and may even be different from a set. Think of this like a class's constructor.
Get (or Retrieve)
This is the generic way to read the data from a value. It retrieves the data value in its default or specified format. For instance~
get number refers to getting the number form of this value even if the default is a string. This is not coercion. Additionally, only references can be retrieved as references.
Set
This is the generic way to write the data to a value. It may be able to accept multiple forms of data but it should generally not be parsing or interpreting anything. The data set in this way should be the same data retrievable by get. This should be able to accept abstract forms as well as loose values.
Coerce
Coercion can only happen between strings and numbers, not lists. If you are coercing something to a string, it can change a number into a string; and if you are coercing something into a number, it can attempt to change a string into a number. This is different from a value having multiple basic data types as these representations are not considered to have the same value. Additionally, when coercing a number into a string, this is not the same as stringification, as prettyprint options are not respected.
Example~
Within the size type, "short" and 2 have the same value. Retrieving the string form of a size value may return "short" and if so, retrieving the number form of the same size value would return 2. However, normally, 2 and "2" do not have the same value but are rather alternate representations of the data which can be coerced into each other.
Convert
Refers to type conversion in the sense of appropriating a certain type (usually a basic type) into a more definitive type as requested by a key's validation scheme. See Types section for more details.
Export
As a verb it refers to the process of converting an abstract form of the data into a standard (or otherwise usable) file format. This format should be configurable by the user and author.
As a noun it refers to those exported files.
Import is the reverse of this process.
A struct which can be exported is called an exportable.
Serialize
As a verb it refers to the process of converting an abstract form of the data into a binary representation.
As a noun, serialized form, refers to that binary representation of the data.
Unserialize is the reverse of this process.
A struct which can be serialized is called a serializable and may be a substruct in a binary context.
Stringify
Converts an abstract form to a string form using whatever prettyprint options are relevant. Typically, a text context will use this to render a struct for use in a string output. The context will handle escapes and quoting itself and may use prettyprint options that the struct specifies to do so. If the context treats this type in a special manner, it will not stingify it, and will prefer to handle it on its own.
A struct which can be stringified is called a stringifiable struct.
Parse
Converts a stringified representation of this value back into an abstract form. This is distinct from Define in that it should only represent the value, much like Set. However this is distinct from Set in that it it must take in a string and that there may be additional information in this form which is ignored as stylization. Parse should, as best it can, try to accept all acceptable formats for this type, regardless of its prettyprint settings. The prettyprint settings should be used when there's a conflict or when there's potential ambiguity for which a warning is insufficient.
Fileable struct
A fileable is a struct which represents a whole virtual or physical file of a particular type (ie. it may not be configured by the author). Its base is a standard path/file type locator where its context could be considered the root directory. This may be used for things such as archive formats or more modern games which are parted into multiple files, even from disc dumps. A fileable struct may often also be a serializable struct and/or provide a context itself.
Reflect
As a verb it indicates that a value is tethered to another value but provides a different outlook on the value. Given B reflects A, this means that when A changes, B must reflect the changes and vise versa. For instance, it may provide access to tokens within a parsed language while the data itself is considered to be the original text.
As a noun, reflection, it refers to one such value that reflects its owner.

Types

There are three basic types: string, numbers, and lists. All values should be able to become at least one of these types.

Interpreted types are those formations which are syntactic sugar for basic types, such as hexnums to numbers. Not all seemingly custom syntaxes are such, though, and are rather conventions for types such as a + prefix for a math value. The given string for a math type key is still only a literal with concern to syntax.

Structs are given more complex types (though one can also define structs of the basic types). These are given explicit types as opposed to having syntactical designations (see struct syntax for more details). These types define what keys are available to them, what their types are, what their valid values are (if restricted), what their default value is (or the fact that it's required), and what substructs are valid. Functionally, it defines if and how it is exportable, serializeable, stringifiable, and/or fileable.

As stated above, keys define what their valid types are. They can accept more than one type. The type given and the type accepted must be compatible in some logical way, regardless of whether or not they're the same or derived from similar types. When retrieving the value of a key, it must return a standardized value. Similarly, when setting the value of a key, it must accept a standardized value (or several) that relates to what it returns. This value must have some logical interaction with the type written by the author. For more details, see the proxying section.

References

When retrieving a value from a reference, before returning the value, it must first validate the value it intends to return in the same manner as a syntactic value would be validated. If the value is to be converted to anything, the conversion should be returned as the value. This is to say that references are lazily evaluated.

Keystructs

These work similarly to references, however they can do conversions immediately if it makes sense. Otherwise it can do a proxy with lazy verification.

Keystructs are also limited to available types that can be declared. This works differently than it does for substructs. Although it does attempt to use types registered to the containing struct first, it will also attempt to use types registered globally.

Hierarchy

Structs in RPL are allowed to exist both in the root and inside of other structs. Within structs there are two types of structs that can be created: substructs and keystructs. Of course each struct also contains regular keys with which to define its contents.

The hierarchy is intended to represent the unification of common values (such as the dimensions of several images) (depth, references), the directory structure of the exported files (depth, naming), and the order of the files in the binary (linear).

Although hierarchy is important to references, that information will be covered in the References section instead.

With regard to unification, key lookups must bubble up up to the top struct, using the nearest value as its own. However the value must be capable of being the requested type. If it is not, or if no ancestor has a key of the requested name, it must use the default value for this key if one is defined. If not, it may attempt to use the default values of parent structs, checking type as it did before. TODO: Flowchart

In order to define a key in a parent struct that will not be seen by substructs which are bubbling up, one may surround that key in curly braces.

Example:

bin {
    {size: 200 bytes}
    number FirstNumber {
        # size defaults to 4, here
        # as defined by the number specification
    }
}

The static struct (and its derivatives) has special handling in this regard. static structs may be declared anywhere, even inside other structs that do not explicitly allow them (and in fact, they should not explicitly allow them). Their valid substructs must be inherited from their parent (if they are in the root then all root's structs are valid substructs). In this way, statics are intended for hierarchical manipulation (in addition to storing static data).

Naming

In order to represent the exported directory structure we resort to struct names. There are up to two names for a struct, as shown in this example:

string StructuralName {
    name: abstract_name
}

The structural name is the name used in making references. The abstract name is the name of the abstract data, particularly if it contains what would be invalid characters in the normal position. However, the name key can also overwrite the abstract structure entirely, as described below.

The name key should use the name type for its value.

When defined, if the string begins with a slash, it is considered to be relative to the CWD (or whatever was passed as this, if such an action is possible), otherwise it is considered to be relative to the parent struct. If this struct is at the root and a slash is the first used character, it is considered to to be relative to the base RPL file's directory. Absolute file locations are not valid.

When retrieved, it must give the absolute hierarchichal location. It should begin at the requested struct. If it has a name key defined, it will use that, otherwise it will use the struct name. If the name key was relative to the parent struct or if it used the struct name, it should continue up to the parent struct and perform the same operation. Each parent represents a containing "folder(s)" of the requested struct such that the result ends up being something like grandparent/parent/requested.

Example:

static This {
    static IsA {
        name: is/a
        exportable File {
            # more on this in Exportables
            exports {
                format: png
            }
        }
    }
}
# @File.name = ${CWD}/This/is/a/File
# @File.name.file = ${CWD}/This/is/a/File.png

static {
    # This struct does not have a name in any form, so it is disregarded for that check.
    # Graphics's file key must then be treated as the root struct in terms of abstract hierarchy.
    blah: "hi"
    static Graphics {
        graphic SomeSprite {
            # ... maybe references @this.blah ...
        }
    }
}
# @SomeSprite.name = ${CWD}/Graphics/SomeSprite
# @SomeSprite.name.file = ${CWD}/Graphics/SomeSprite.png
# Note that png is the default extension for graphics structs.

It should be recommended to RPL authors that they represent exports in this organized manner, particularly by defaulting to the struct names where possible. However, it may not be wholly feasible, and these are the cases where the name keys are helpful. Even in these cases, though, it should be recommended to use a structural name that relates enough to the abstract name to be easily identifiable.

Binary order

Serializable structs must always define a key called base which gives the location of the data in the binary. It must use the address type. If there can be multiple base locations in the binary, it may represent this as keys named base1, base2, etc. base should not redirect to any of these in this case, it should not be defined.

When defined, in a binary context, it can take a number or a number and an indicator of relativity. When only a number is given, the relativity indicator is assumed to be the beginning of the file. The other two indicators are the current location and the end of the file (in which case, the number given is treated as a negative). For more information on what the labels are, see the address type definition.

When retrieved, it returns the absolute address in the context. If the key was not defined, it defaults to 0, relative to the current location. The current location is just after the end of the previous struct, reading top to bottom. Substructs contribute to the length of a struct. When the base of a struct needs to be determined, this younger sister asks its next older sister (the struct positioned directly above it, linearly, within the same depth) what its end is, which is generally its base + its length. If it doesn't have an older sister it asks its parent what its base is. The younger sister (or eldest child) then returns that value + its offset as its own address. If the very first struct in a context (the root is also a context) is relative to the current location, it must also be relative to the beginning of the context. If an older sister is not exportable, the next older sister must be asked instead, and so forth.

Text order

When dealing with a textual context (see more in Text and Bin section), the base key takes a string or a string and an indicator of relativity.

The string used is generally a selector of some sort, however its format differs based on the type of container. The syntax will be specified in the section for the container itself. The indicator of relativity can differ based on context as well, and one may not always be available.

When retrieved, if this address is relative to the root, it should return the selector as-is. If it was relative to the parent, it should combine the relevant selectors as appropriate for the format and return that in order to provide an absolute selector. This sort of concept could be applied to forms of relativity besides descendants as well, such as sibling relativity (eg. in CSS, combining relevant selectors with " + ").

Specialized keystructs

Sometimes a key may say it takes a specialized keystruct. This is a key which can be defined in the following ways, using "test" as the key in question:

When defined in the basic form, it instantiates a struct of type test, using the syntactic form give for defining it. This is the normal behavior for a key of test type.

When defining the substruct, it instantiates the struct as it would a substruct, then associates the result with the test key. If the author defines a name here, it may also be stored as a child, depending on implementation. Despite being associated with a key, this form does not allow substructs to see it when bubbling, as it was not explicitely defined as a key. For example:

tester Tester {
    test { ... }

    tester ChildTester {
        # Does not have a test key, probably throws an error!
    }
}

The last method is a way to allow inheritance while also defining the specifics that cannot be addressed from a basic form.

Internationalization

RPL should support any reasonable script and language. Unreasonable scripts refers to only really archaic or constructed ones with unreasonable demands. For instance, Lojban's quotations may be unfeasible and it is not given much priority since it is not a natural language.

Ideally, syntactic constructions should support natural ideologies surrounding the punctuation used. For instance, (though I'm not Japanese nor have I asked a native speaker about it) I've supported hyphens as traversal notation for references because the hyphen can also be pronounced as の "no", the possessive particle, in Japanese (eg. in phone numbers). However syntactic indicators should indeed always be some sort of symbol and not a letter class character.

There may be locality definitions (in a header struct) for certain features, such as dates, which can be incredibly ambiguous. These should be definable per feature (eg. date, time, …) as well as generally by simply specifying a locale. Per-feature definitions supersede the general locale.

Since type names and key names have specific meanings tied to their implementations they should be translatable. The libraries themselves can be written in any language so there is no "default language", the i18n files will just key off of what the library implements. Imperial, then, should be able to take any RPL file and translate it into the reader's language and locale regardless of the author's.

i18n should also be able to translate errors, standard terms (eg. Defs, signed, etc), help contents, and so forth.

Text and Bin

Some struct types create new data contexts from which their substructs are based. A struct that uses such a type is called the container. They can create binary contexts, textual contexts, or *file contexts which differentiate how their substructs address the data within them. Binary contexts are always addressed by the address, in bytes, within the context. The beginning of the context (not the container itself) has the address of 0.

The basic binary container type is bin. This is the type that the root implicitly uses when interpreting a binary file. It provides access to raw binary data and is generally redundant. See the bin section in Basic Structures for more information.

One could consider the basic textual container to be string but it offers little functionality. Rather, there is a generic textual container called markup. See the markup section in Standard Structures for more information.

The basic file container is folder. This is the type that the root implicitly uses when interpreting a folder. It provides access to a directory on the actual file system. There is no ability to define one outside of a folder context as there is no way to refer to absolute locations in the file system.

There are other binary containers that will generally deal with encryption and compression schemes. Textual containers will generally deal with parsing object markup languages. File containers will generally deal with archives like zip.

Containers have the ability to accept any struct as a substruct which is able to un/pack the kind of data it expects (or is a static which contains such things). All substructs within this container and all descendants therein regard the nearest parent container as if it were the root in terms of addressing (however, bubbling up key fetches and parental addressing must still be possible).

The un/packing functions required by each container are:

Containers do not necessarily need to have substructs in them. Textual contexts will generally be able to be exported and referenced as if the were strings and binary contexts as if they were bins. However, if the substructs manage all their data, it can no longer export itself.

All textual contexts should be able to define an encoding. The default, however, is not US-ASCII but rather "unencoded", which makes it essentially equivalent to a bin (consider it a string which only contains the codepoints 000000FF). Certain types may also be able to determine the encoding on their own and should do so if they can but only in the case of the encoding defaulting (it's allowed to provide a warning if the requested type and the internal type don't match, though). Additionally, if a textual container is defined in a textual context, it will not use the encoding settings (as the text has already been decoded).

References & Resolution

See the syntax section for the reference syntax. Here, we will be referring to the components of a reference: the struct, the keys, and indexes.

Libraries should not have to know they're dealing with references. When a reference is used, the value will not be validated against or recasted to the key's type and valid values until the value is requested.

One can reference only a struct, a substruct in a struct, a key in a struct, or an index in a list. In few locations, referencing a struct may regard it as a reference to a struct. When referencing a key that contains a reference to a struct in this way, it is also considered to be a reference to a struct. You may only refer to keys when referring to a struct (keep in mind that all values are also structs).

When a value is retrieved from a struct, it uses what are called basic values. They map to the basic types: string, number, and list. Not all types will return a value for each basic type. Some structs may not return a value for any of them. However, all structs regarded as values will. Simply put, if a struct represents a string, fetching its string value will return that string. Therefore, when placing a reference to this struct in a key that has a string type, this new key will have the value of that string.

Example:

static A { thing: size (byte) }

string String { data: @A.thing } # @String.data = "byte"
number Number { data: @A.thing } # @Number.data = 1
static Static {
    ref: @A.thing # @Static.ref is just a reference
    str: @String  # @Static.str = "byte"
}
number Number2 { data: @Static.ref } # @Number2.data = 1
number Number3 { data: @Static.str } # Throws a type exception error; number type can't be a string

Simple form

This is a "strict" style of representation. Things in this form are meant to be simple to remember, with no ambiguity. It may only contain the following Unicode character classes: Letter, Lowercase; Letter, Other; Letter, Modifier (possibly); Number, Decimal (at the end only). However not everything in these classes are permitted, for instance only normal forms are permitted (ie. 0 is permitted but 𝟬 may not be unless it has the same meaning). Certain exceptions could be made, such as allowing scientific notation for the end (ie. 10e2), but not for allowing 2 to represent the word "to".

Simple forms should only be one word. It is best if the word is common but specific. If it is representing a concept that could easily inspire several words (for instance "upgrade" and "update") it is best that appropriate aliases be used so as to not leave the sister forms available for distinct meanings and subsequent confusion. If you must write two words, consider the first word be a key to a keystruct which contains the second. If you are certain that it's a single concept (ie. nothing else would go in this subtruct) and there's no means of using only a single word, at least ensure that it's as easy to read as possible.

Easy to write

RPL is intended to be easy to write, which basically means that it has to be easy to remember. Although there should be ample help available for how to write it, it should also only take a very basic level of familiarity to get started. Part of this is relieved by enforcing simple form ('Was it underscores or camel case…?') and the other portion of it should be by using common, single-word terminology for keys and types and by associating alternative words and word forms with the same key names. This may extend to types but likely only in word forms rather than synonyms.

Much of what can be done to achieve this goal is locale-specific, so this section is only relevant to English-language locales.

At least if a key is able to take a list (and they should, if it makes sense), a key name should always alias its plural (or singular, if plural is the default). For instance, in the RPL struct one can use lib or libs to refer to the lib key, regardless of whether or not it's taking multiple values. However, if possible, it would be best for the i18n to choose the appropriate key name based on the contents when recreating a struct as a string (eg. in making templates or dumping debug info). Despite that, it really should not be anal about it, for example, datum may be the singular of data but datum is not a remotely common word and should never be preferred.

Syntax should be easy to remember, practical, familiar, etc. Keys should accept multiple types where it makes sense, essentially it should accept any natural representation (eg. size accept both 5 and 5 bytes as the same value). Literals are sort of the exception to this rule since they have such restrictions but if one doesn't abuse them for general strings it should generally be okay and syntax highlighters can deal with alerting users to issues. Custom syntaxes (parsed out of literals or strings) should generally be discouraged, especially if RPL is up to the task, or at least not be the only option. The glaring exception to this is existing standards, like math and what's used in textual context's bases as selectors. Similarly, lists that are used like arguments for a declaration should never have more than two arguments whose meaning relies on position. Both of these things can beget faulty recollection.

Additionally, RPL has the principle of assuming that an author means what they write (syntax errors aside). Any automation and value resolution should restrict itself to working within what the author has explicitly written, without changing those values. For instance, if a string does not have data but an author has explicitly given it a length, all entries must fit within that length. The structure the author provide is called the rigid structure.

Structure and sources

An RPL implementation should track the source of data, as well. Some possible types could be:

With this, and all the above sections combined, we can imagine the correlation between what the author writes and what the internal structure looks like as something like this:

# Types #
number int { size: 4 }
string filename { size: 12 }

# Structure #
string Magic { data: "HELLO" }

data Header {
    xnumfiles: [number, 4]
    xoffset1: [number, 4]
    xoffset2: [number, 4]
    xnames: [@filename, @this.xnumfiles, @this.xoffset1]
    xfiles: [@int, @this.xnumfiles, @this.xoffset2]
}

graphic TestImage {
    name: @Header.xnames[0]
    base: @Header.xfiles[0]
    end: @Header.xfiles[1]

    width: 20
    read: LRUD
    pixel: 0bRGB
}

It may be useful to understand the data and graphic structs first.

It is important to understand how these references affect the interpretation, too. The data struct, by its definition, takes a type as the first element of an x-key's list. Thus, when supplying a reference instead, it interprets the target as a type. This forces the target to be removed from the context in terms of serialization and relative basing. All other references are relatively regular ones, they establish a link between the referring key and its referrant. That is, they contain the same value.

If we expand this to how it would be interpreted (ignoring exports and pretty):

# Since these are used as types for data x-keys,
# only their rigid structure matters.
# This is the type and the size, for both.
number int { size: 4 }
string filename { size: 12 }

# Structure #
string Magic {
    base: $000000         # source: defaulted/implied
    name: Magic           # source: implied by struct name
    tags: []              # source: defaulted

    data: "HELLO"         # source: rigid
    length: 5             # source: implied by data
    size: 5               # source: implied by data + encoding
    encoding: unencoded   # source: defaulted
    # note that this is assuming unicode 0000~00FF
    # or whatever python assumes for its bytes repr

    padding: $00          # source: defaulted
    align: left           # source: defaulted
    padside: right        # source: defaulted

    end: math(@this.base  # source: calulated
        + @this.size)
}

data Header {
    base: @Magic.end      # source: implied by linear ordering
    name: Header          # source: implied by struct name
    tags: []              # source: defaulted

    sign: unsigned        # source: defaulted
    endian: little        # source: defaulted
    padding: $00          # source: defaulted
    align: left           # source: defaulted
    padside: right        # source: defaulted

    # comment is undefined by default

    # Here we're showing what's instantiated.
    xnumfiles: number {             # source: rigid
        base: @parent.base          # source: implied by x-key order
        # name and tags are not defined in keystructs
        # data and bits are not currently defined,
        # because the data hasn't been imported

        size: 4                     # source: rigid
        sign: @parent.sign          # source: "inherited"
        endiant: @parent.endiant    # source: "inherited"

        end: math(@this.base        # source: calculated
            + @this.size)
    }

    xoffset1: number {              # source: rigid
        base: @parent.xnumfiles.end # source: implied by x-key order

        size: 4                     # source: rigid
        sign: @parent.sign          # source: "inherited"
        endian: @parent.endian      # source: "inherited"

        end: math(@this.base        # source: calculated
            + @this.size)
    }

    xoffset2: number {              # source: rigid
        base: @parent.xoffset1.end  # source: implied by x-key order

        size: 4                     # source: rigid
        sign: @parent.sign          # source: "inherited"
        endian: @parent.endian      # source: "inherited"

        end: math(@this.base        # source: calculated
            + @this.size)
    }

    xnames: list {                  # source: rigid
        base: @this.xoffset1        # source: rigid

        type: string {              # source: rigid, filename
            # base, name, and tags are irrelevant, here
            # since there is no data, there is no length either
            size: 12                # source: rigid, filename
            encoding: unencoded     # source: defaulted

            padding: $00            # source: defaulted
            align: left             # source: defaulted
            padside: right          # source: defaulted
        }

        length: @this.xnumfiles     # source: rigid

        end: math(@this.base        # source: calculated
            + @this.type.size * @this.length)
    }

    xfiles: list {                  # source: rigid
        base: @this.xoffset2        # source: rigid

        type: number {              # source: rigid, int
            size: 4                 # source: rigid, int

            sign: unsigned          # source: defaulted
            endian: little          # source: defaulted
        }

        length: @this.xnumfiles     # source: rigid

        end: math(@this.base        # source: calculated
            + @this.type.size * @this.length)
    }

    end: @this.xfiles.end           # source: implied
}

graphic TestImage {
    base: @Header.xfiles[0] # source: rigid
    name: @Header.xnames[0] # source: rigid
    tags: []                # source: defaulted

    size: math(@this.end    # source: calculated
        - @this.base)
    width: 20
    height: math(@this.size # source: calculated
        / (@this.width * @this.pixel.size))
    dimensions: [
        @this.width,        # source: implied
        @this.height        # source: defaulted
    ]

    x: 0                    # source: defaulted
    y: 0                    # source: defaulted
    offset: [
        @this.x,            # source: defaulted
        @this.y             # source: defaulted
    ]

    read: readdir(LRUD)     # source: rigid
    pixel: pixel {          # source: rigid
        size: 3 bits        # source: calculated from format
        format: 0bRGB       # source: rigid
        order: forward      # source: defaulted
        chunk: forward      # source: defaulted
        rgb: [0, 0, 0]      # source: defaulted
        # hsl ignored for being ill-defined
        alpha: 100          # source: defaulted
    }

    palette: []             # source: defaulted
    # data, rows, and other channels are not currently defined,
    # because the data hasn't been imported

    rotate: 0               # source: defaulted
    mirror: false           # source: defaulted
    flip: false             # source: defaulted
    blank: transparent      # source: defaulted

    end: @Header.xfiles[1]  # source: rigid
}

The references link the structure in the following manner:

If anything changes, such as by the application altering values, all of these established formulae must still hold true. If something occurs where they cannot hold true, it must raise some sort of error. For instance, if one attempts to set a value to TestImage.size that is not equal to the calculated value, an error must be raised, because both base and end are rigidly set.