Overhaul the build system (#234)

* wip

* fix

* add disassembler

* Disasm builds OK

* Variable addends

* More wip

* Rodata migration implemented

* Cleanup old tools

* Try fix submodule -> subrepo merge

* git subrepo pull --force --remote=https://github.com/zeldaret/ZAPD.git tools/ZAPD

subrepo:
  subdir:   "tools/ZAPD"
  merged:   "602e609"
upstream:
  origin:   "https://github.com/zeldaret/ZAPD.git"
  branch:   "master"
  commit:   "602e609"
git-subrepo:
  version:  "0.4.3"
  origin:   "https://github.com/ingydotnet/git-subrepo"
  commit:   "2f68596"

* Builds again but assets are totally broken

* git subrepo pull --force tools/asm-processor

subrepo:
  subdir:   "tools/asm-processor"
  merged:   "1ffdb08a"
upstream:
  origin:   "https://github.com/simonlindholm/asm-processor.git"
  branch:   "master"
  commit:   "1ffdb08a"
git-subrepo:
  version:  "0.4.3"
  origin:   "https://github.com/ingydotnet/git-subrepo"
  commit:   "2f68596"

* More cleanup, move functions.txt and variables.txt to tools/disasm and rm tables

* rm z64compress in preparation for subrepo

* git subrepo clone (merge) https://github.com/z64me/z64compress.git tools/z64compress

subrepo:
  subdir:   "tools/z64compress"
  merged:   "eb11085c"
upstream:
  origin:   "https://github.com/z64me/z64compress.git"
  branch:   "main"
  commit:   "eb11085c"
git-subrepo:
  version:  "0.4.3"
  origin:   "https://github.com/ingydotnet/git-subrepo"
  commit:   "2f68596"

* Fix asset extraction

* Fix diff-init make rule

* Split code bss

* Split assumed linker bug padding from assembly files

* add filelists for mm.us.rev1

* Maybe working, but I'm not sure

* add overlays to spec

* Add rodata to actos

* Everything compiles

* Make a lot of C files for code

* Add almost every file in code to spec

* whoops

* 3 code files left

* add scenes to spec

* More progress on progress.py

* Fix skelanime in spec

* audio files!

* Fix merge issues

* Fix some C files in code

* Fix remaining code files

* Use existing O1 C files in spec

* reorder boot order in spec

* update spec

* fault.c

* Convert relocs on completed actors, fixbaserom uses current rom name

* more boot files

* Add VT macros and script

* finish already existing boot files

* most of  libultra

* fix 64bits libultra files

* Use C files for libultra, wrap some functions in NON_MATCHING

* Remove duplicate of OS_CLOCK_RATE from fault.c

* C files for fbdemos

* delete dumb files

* bootstrap C files, still need to add them to the spec

* update fixbaserom

* boot OK?

* I forgot to commit the spec

* C for gamestates

* C for kaleido

* Change all includes to ""

* copy actor sizes script from oot

* I forgot to delete those files

* Basic C files for effects

* Add effects initvars names

* Remove mislabelled boot functions from header/txt

* Begin porting bootstrap_fx, some sizes

* Fix <>

* Fix enum

* Fix diff.py

* fix libultra stuff

* update regconvert

* update setup warnings

* add some missing ;

* Fix some makefile stuff and other fixes on some non_matching functions

* add executable flag in extract_baserom and fixbaserom

* fix relative path

* copy assist from oot

* fix map path

* another assist path fix

* Delete C files for handwritten files

* add code_801A51F0 to spec

* add gfxbuffers to spec

* Move rodata to top of each file when possible

* UNK_TYPEs for func_801A51F0

* Remove kaleido rodata from spec

* Update spec and undefined_syms for recent merge

* GCC warnings and fix errors in nonmatchings,

* round percentage numbers

* progress script: format changes

* progress: error on non-existing files

* fix warning in z_scene_table

* Match 2 nonmatchings in z_actor

* Warnings in lightswitch and invadepoh

* Fix warning in z_actor_dlftbls

* I though I fixed this one

* whoops

* Comment out CC_CHECK

* Removed redundant ultra64.h includes

* Update asm_processor, sorted boot_O1 into other folders, completed the fbdemo bootstrap, cleaned up undefined_syms

* Completed gamestates bootstrap

* Split kaleido_scope

* Remove section.h and segment.h, move keep object externs to a common location in variables.h

* Completed effects bootstrap

* Segmented address externs for effects, fbdemos, gamestates and kaleido

* Move actor data externs out of the if 0

* Segmented address externs for actors

* Prepare actionfunc detection

* fix script, how did it even work before

* Fix actionfunc script again, re-introduce some more intermediate prints to the disassembler

* Automated actionFunc detection in actors

* Segmented addresses from player .text

* rm old segment addrs script and fix build

* Move sizes folder to tools

* Make build.py executable

* New Jenkinsfile Prayge

* Remove numpy dependencies

* Add warnings_disasm_current.txt

* my bad

* Update spec and undefined_syms

* Add z_eff_ss_hahen to pametfrog

* git subrepo pull (merge) --force tools/z64compress

subrepo:
  subdir:   "tools/z64compress"
  merged:   "163ca2af"
upstream:
  origin:   "https://github.com/z64me/z64compress.git"
  branch:   "main"
  commit:   "163ca2af"
git-subrepo:
  version:  "0.4.3"
  origin:   "https://github.com/ingydotnet/git-subrepo"
  commit:   "2f68596"

* Make z64compress print to stdout

* sneeky commit to update warnings tooling

* test

* Another test

* Mark fixing overlay reloc generating as a TODO

* Update warnings stuff

* Communicate the return code from running z64compress back to the Makefile through the wrapper

* Run formatter, remove extra commented copy of function

* Re-fix some includes

* Convert atan to hex to conform to decided style

* Some tidying up, remove c for fp and the other two handwritten code files

* BSS in z_collision_check & z_scene_proc

* add static back in

* Fix timerintr bss, add file to spec, some cleanup

* Remove externs

* Newline

* Readd enums

* Typo

* Colours

* Comments for hitmark enum values

Co-authored-by: EllipticEllipsis <73679967+EllipticEllipsis@users.noreply.github.com>

* Improvements and suggestions

* Organize and remove unused imports and use env for python3 scripts, delete unused overlay.py

Co-authored-by: angie <angheloalf95@gmail.com>
Co-authored-by: Elliptic Ellipsis <elliptic.ellipsis@gmail.com>
Co-authored-by: engineer124 <engineer124engineer124@gmail.com>
Co-authored-by: EllipticEllipsis <73679967+EllipticEllipsis@users.noreply.github.com>

1 files changed, 88 insertions, 5 deletions

diff --git a/tools/regconvert.py b/tools/regconvert.py
index fdc1090b6..f4b9b0db1 100755
--- a/tools/regconvert.py
+++ b/tools/regconvert.py
@@ -1,6 +1,6 @@
 #!/usr/bin/env python3
 
-import argparse
+import argparse, re
 
 GROUP_SIZE = 0x60
 DATA_OFFSET = 0x14
@@ -11,13 +11,96 @@ def get_reg_macro(offset):
     reg = (offset - DATA_OFFSET) // 2
     group = reg // GROUP_SIZE
     reg_in_group = reg % GROUP_SIZE
-    return "%cREG(%d)\n" % (REGISTER_NAMES[group], reg_in_group)
+    return "%cREG(%d)" % (REGISTER_NAMES[group], reg_in_group)
+
+def parse_number(string, force_hex):
+    if string.lower().startswith("0x") or force_hex:
+        return int(string, 16)
+    return int(string, 0)
+
+def index_to_offset(index):
+    return index * 2 + DATA_OFFSET
+
+def read_file(filename):
+    parsed_contents = ""
+
+    src_file = open(filename)
+    file_contents = src_file.read()
+    src_file.close()
+
+    pattern = re.compile("gGameInfo->data\[((0[xX])?[0-9a-fA-F]+)\]")
+
+    match = pattern.search(file_contents)
+    while match:
+        index = parse_number(match.group(1), False)
+        offset = index_to_offset(index)
+
+        start, end = match.span()
+        parsed_contents += file_contents[:start]
+        parsed_contents += get_reg_macro(offset)
+
+        file_contents = file_contents[end:]
+        match = pattern.search(file_contents)
+    
+    parsed_contents += file_contents
+    return parsed_contents
+
+def write_file(filename, contents):
+    with open(filename, "w") as f:
+        f.write(contents)
+
+def check_valid_index(index):
+    return 0x0 <= index <= 0xADF
+
+def check_valid_offset(offset):
+    return 0x14 <= offset <= 0x15D2
 
 def main():
-    parser = argparse.ArgumentParser(description="Converts a GameInfo offset to a REG macro.")
-    parser.add_argument("offset", help="offset to GameInfo in hexadecimal")
+    parser = argparse.ArgumentParser(description="Converts a gGameInfo->data index to a REG macro.")
+
+    # TODO: Add a description and a better title
+    # index_group = parser.add_argument_group(title="index", description="")
+    index_group = parser.add_argument_group()
+    index_group.add_argument("index", help="index of gGameInfo->data in decimal (or hexadecimal if starts with 0x)")
+    index_group.add_argument("--offset", help="treat index argument as an offset to gGameInfo instead", action="store_true")
+    index_group.add_argument("--hex", help="treat index argument as hexadecimal, even without the 0x", action="store_true")
+
+    # TODO: Add a description and a better title
+    # file_group = parser.add_argument_group(title="file", description="")
+    file_group = parser.add_argument_group()
+    file_group.add_argument("--file", help="use the parameter as a filepath, then replace every occurrence of `gGameInfo->data[number]` in that file a REG macro", action="store_true")
+    file_group.add_argument("--stdout", help="print the processed file to stdout instead of overwriting the original", action="store_true")
     args = parser.parse_args()
-    print(get_reg_macro(int(args.offset, 16)))
+
+    if args.file:
+        filename = args.index
+        contents = read_file(filename)
+        if args.stdout:
+            print(contents)
+        else:
+            write_file(filename, contents)
+    else:
+        index = parse_number(args.index, args.hex)        
+        if args.offset:
+            offset = index
+            if not check_valid_offset(offset):
+                print("Invalid offset: " + args.index)
+                if args.hex:
+                    print("Offset should be in the range from 0x14 to 0x15D2")
+                else:
+                    print("Offset should be in the range from 20 to 5586")
+                exit(-1)
+        else:
+            if not check_valid_index(index):
+                print("Invalid index: " + args.index)
+                if args.hex:
+                    print("Index should be in the range from 0x0 to 0xADF")
+                else:
+                    print("Index should be in the range from 0 to 2783")
+                exit(-1)
+            offset = index_to_offset(index)
+
+        print(get_reg_macro(offset)+"\n")
 
 if __name__ == "__main__":
     main()