Fixed-width files and COBOL record layouts

Most RRC datasets are fixed-width: no delimiters, no header row, no types. A record is a fixed number of bytes and every field is a byte range. The only documentation is a record layout, published as a PDF, written in the vocabulary of COBOL.

Once you can read one of those layouts you can parse any of these files. There are about five concepts to learn, and then it is mechanical.

The layout document

An RRC layout lists, for each field: a name, a starting position, a length, and a PIC clause. Positions are 1-based, counting from the first byte of the record, which means every slice you write in a 0-based language needs a shift of one:

def get_field(name, pos, length, line):
    return line[pos - 1 : pos + length - 1]

Get that off by one and every field is shifted by a character. The symptom is distinctive: numbers that are consistently a factor of ten out, and text fields that all begin with the last letter of the previous field.

Layouts also have FILLER entries — reserved or unused bytes. They are part of the record length and must be counted, even though there is nothing in them. A parser that skips filler while computing positions will drift.

PIC clauses

The PIC (picture) clause declares a field's shape.

PIC X(32) — 32 alphanumeric characters. Text. Space-padded on the right.
PIC 9(05) — five digits. Zero-padded on the left.
PIC S9(05) — five digits, signed. See below; this is the interesting one.
PIC 9(03)V9(07) — ten digits with an implied decimal point after the third.
PIC S9(08)V99 — eight digits, a decimal point, two more digits, signed.

The V is the whole trick: it marks where the decimal point would be. It occupies no byte. A PIC 9(3)V9(7) field is ten characters wide and the file contains 0312345678, which means 031.2345678.

There is no way to detect this from the data. A ten-digit run of characters looks the same whether it is an integer or a scaled decimal. You have to read the layout. This is the second most common source of wrong numbers in RRC data, and the first is the sign encoding.

Widths and decimals are worth carrying through your whole pipeline. EZRRC records position, width and decimals for every fixed-width column and shows them on the dataset page, precisely so you can check a value against the original bytes.

Zoned decimal: why numbers end in `}` or `J`

Open a fixed-width RRC file and you will eventually find a numeric field that ends in a letter:

0321234567C
00000000123}

This is a zoned decimal — the standard COBOL representation of a signed number in character data. The sign is not a separate character. It is overpunched onto the final digit, encoding two pieces of information in one byte.

Recall from the EBCDIC article that a character's high nibble is its zone. In EBCDIC, digits 0–9 are F0–F9. A signed field replaces the zone of the last digit: C for positive, D for negative. So positive one is C1 and negative one is D1.

And C1 in EBCDIC is the letter A. D1 is J. C0 is {. D0 is }.

That is the entire mystery. The trailing character is a digit whose sign has been written into its top four bits, and after conversion to ASCII you see the letter that occupies that byte value.

Last character	Digit	Sign
`{`	0	+
`A` `B` `C` `D` `E` `F` `G` `H` `I`	1–9	+
`}`	0	−
`J` `K` `L` `M` `N` `O` `P` `Q` `R`	1–9	−

So 00000000123} is −1230, not "1230 with a brace on it". And 0321234567C carries a final digit of 3 and a positive sign.

Decoding is straightforward once you know:

COBOL_POSITIVE = {'{': '0', 'A': '1', 'B': '2', 'C': '3', 'D': '4',
                  'E': '5', 'F': '6', 'G': '7', 'H': '8', 'I': '9'}
COBOL_NEGATIVE = {'}': '0', 'J': '1', 'K': '2', 'L': '3', 'M': '4',
                  'N': '5', 'O': '6', 'P': '7', 'Q': '8', 'R': '9'}


def decode_cobol_signed(val, integer_digits=3, decimal_digits=7):
    """Decode a COBOL signed zoned decimal (S9(3)V9(7)) to a float."""
    if not val or len(val) < 2:
        return None

    last_char, digits = val[-1], val[:-1]

    if last_char in COBOL_POSITIVE:
        sign, digits = 1, digits + COBOL_POSITIVE[last_char]
    elif last_char in COBOL_NEGATIVE:
        sign, digits = -1, digits + COBOL_NEGATIVE[last_char]
    elif last_char.isdigit():
        sign, digits = 1, digits + last_char      # unsigned zone: F, positive
    else:
        return None

    if not digits.isdigit():
        return None

    padded = digits.zfill(integer_digits + decimal_digits)
    whole = padded[:integer_digits]
    fraction = padded[integer_digits:]
    return sign * float(f"{whole}.{fraction}")

Note the third branch. A field declared signed is not obliged to carry a signed zone; an unsigned F zone renders as an ordinary digit and means positive. Handle all three cases or you will null out most of your rows.

Where this bites in practice

The Full Wellbore Database stores WGS84 latitude and longitude as PIC S9(3)V9(7) — ten characters, seven implied decimal places, zoned sign. A parser that reads them as integers produces coordinates like 312604028. A parser that inserts the decimal but ignores the sign produces a Texas longitude of +102.28, which is in China.

Because Texas is entirely in the western hemisphere, the upstream loader corrects a positive longitude defensively:

lat = decode_cobol_signed(loc.get('wgs84_latitude'))
lon = decode_cobol_signed(loc.get('wgs84_longitude'))
if lon and lon > 0:
    lon = -lon      # Texas longitudes are always negative

That is a domain assumption, and it is stated openly rather than hidden. Be suspicious of any coordinate pipeline that does not tell you what it assumed.

Dates, split across fields

COBOL predates the idea of a date type. RRC layouts represent dates two ways.

As a single CCYYMMDD numeric field. Eight digits: century, year, month, day. 20260730. Parse with %Y%m%d, and treat 00000000 as null.

As separate century, year, month and day fields. The wellbore root record has orig_compl_cc, orig_compl_cent, orig_compl_yy, orig_compl_mm and orig_compl_dd as five distinct fields. You reassemble them:

def assemble_date(cc, yy, mm, dd=None):
    if cc and yy and mm and int(cc) > 0 and int(mm) > 0:
        year = int(cc) * 100 + int(yy)
        day = int(dd) if dd and int(dd) > 0 else 1
        return datetime(year, int(mm), min(day, 28)).date()
    return None

Note the two defensive moves. A missing day becomes the first of the month rather than a null date, because in this data a zero day usually means "we know the month and not the day." And the day is clamped, because a record can carry a day number that is not valid for its month — the mainframe never validated it.

Historical records are the worst offenders: partial dates, zero months, and century codes that distinguish 19xx from 20xx precisely because the field was designed when two-digit years were normal.

Sentinels

Fixed-width numeric fields cannot be null, so "no value" is encoded as a value. The conventions you will meet:

All zeros means absent. A five-digit depth of 00000 is not a well at the surface, it is a well with no recorded depth. Upstream treats an all-zero numeric field as None on read.
All nines occasionally means "unknown" or "maximum" in older records.
Spaces in a numeric field mean absent.
00000000 in a date means absent.

Convert every one of these to null on load. If you do not, your minimum depth is zero, your earliest date is the year zero, and your averages are meaningless.

Turning a layout into a parser

The whole process, in order:

Get the layout PDF. Note the total record length and confirm it against the file size — a fixed-length file's size should be an exact multiple.
Transcribe the fields as (name, position, length, type) tuples. Include filler so the positions stay honest.
Check whether the file is one record type or several. Hierarchical files carry a record type in the first bytes; the Full Wellbore Database uses the first two bytes and has 28 of them.
Slice, strip, and convert: text gets stripped, numerics get sentinel handling, signed fields get zoned-decimal decoding, dates get assembled.
Validate against something you know. Pick a well you can look up on the RRC's own site and confirm the parsed values match.

EZRRC publishes step 2 for every fixed-width dataset it carries: every column's position, width, implied decimals and source name are on the dataset page and in the API, transcribed from the RRC manual. You can use them to write your own parser against the pristine original, which is exactly the point of keeping the original.

Next: production data — the largest and most misunderstood dataset in the collection.