summaryrefslogtreecommitdiff
path: root/tools
AgeCommit message (Collapse)Author
21 hoursspec: change `include` to ignore duped includes across spec (#2777)HEADmainDragorn421
* spec: impl `include_once` and error on duped includes across spec * duped includes fixups * rm get_single_segment_by_name from spec.c (unused code) * replace include logic with include_once's
46 hours[rsp] Introduce microcode build process, disassemble and build both rspboot ↵Tharo
versions (#2764) * [rsp] Introduce microcode build process, disassemble and build both rspboot versions * Suggested changes * Check-in tool for generating armips.cpp from upstream sources, regen armips.cpp against a newer commit * License armips_gen.py, add notice that armips.cpp is automatically generated * Qualify comment with X105 * Clarify the assumed purpose of rspboot releasing SP_SEMAPHORE
4 daysFix fix_bss.py tiebreaking algorithm (#2779)cadmic
2026-05-09Assets system: reconcile xml versions using implicit and relative offsets ↵Dragorn421
(#2614) * Assets system: reconcile xml versions using implicit offsets * simplify * remove other object_mag xmls * Introduce relative offsets: `Offset=".+0x421"`, more sizable `ResourceDesc`s, update spec * fix * reconcile object_mo xml * reconcile object_tw xml * reconcile object_xc xml (and add Tlut="gNameTLUT" to Texture resources) * reconcile object_fd xml * reconcile object_kingdodongo xml * bss * reconcile gameplay_keep.xml
2026-05-08Compiler binaries download: separate from extract and checksum (#2753)Dragorn421
2026-05-06Partial linking of spec segments (#2661)Tharo
* Partial linking of overlay segments, relax linker script alignment * Partial linking of all spec segments * Fix update, remove _RomPos from linker script * iQue version working pending COM-plugin update * Add plf map file resolution to sym_info.py, local symbol merging is broken * git subrepo pull tools/com-plugin subrepo: subdir: "tools/com-plugin" merged: "c4f3ba845" upstream: origin: "git@github.com:Thar0/com-plugin.git" branch: "main" commit: "c4f3ba845" git-subrepo: version: "0.4.6" origin: "https://github.com/ingydotnet/git-subrepo" commit: "110b9eb" * Make tools compatible with partial linking Co-authored-by: Anghelo Carvajal <angheloalf95@gmail.com> * Remove unused files * Fix some makefile bits * mkspecrules cleanup * Comment on the makerom linker layout in mkldscript * Revert linker padding strategy back to pad_text spec directives * Comment on objcopy elf -> rom step * Adjust tool descriptions * Fix compressed builds * rm reloc_prereq, no longer used * Makefile merge fix Co-authored-by: Dragorn421 <Dragorn421@users.noreply.github.com> --------- Co-authored-by: Anghelo Carvajal <angheloalf95@gmail.com> Co-authored-by: Dragorn421 <Dragorn421@users.noreply.github.com>
2026-05-06Remove makeromfs (#2752)Tharo
This is a tool introduced a long time ago that ended up not being used at any point in the build system and has no prospects of future use. There's no reason for keeping it around.
2026-04-29Migrate CI to Github Actions (#2742)Dragorn421
* Migrate CI to Github Actions (#2740) Changes the CI system from Jenkins to Github Actions (GHA) as discussed on discord. I tried to adapt the original jenkinsfile as I could. One major difference from the old system is each version will now be built in parallel instead of sequentially. Also, the mapfiles will be uploaded as an GHA artifact to the Action workflow. I think having them easily available would be nice. This specific approach to handle GHA for decomp projects was adapted from the one used by the GC/Wii community. It is documented here https://github.com/encounter/dtk-template/blob/main/docs/github_actions.md There's a writeup about this adaptation for N64 projects [here](https://github.com/AngheloAlf/drmario64/pull/19). * Rename generate_patch_from_gha -> gha_fix_bss_and_generate_patch * Make check formatting always on the full repo * ruin z_fishing bss for CI testing purposes * fix not passing VERSION to tools/gha_fix_bss_and_generate_patch.sh * debugging gha_fix_bss_and_generate_patch.sh * fix the debugging... * git config safe.directory * rm debugging stuff from gha_fix_bss_and_generate_patch.sh * fix_bss gha-side machinery, attempt 1 * fix1 * fix2 * debug1 * checkout repo in merge_bss_fixes job * fix3 * fix4 * fix5 * some prettify * apply_fix_bss_patches.py complete * fix6 * generate_patch.sh is back * apply fix bss changes from gha! it works! * generate_patch.sh suggestion * ruin bss again for testing * unruin bss * Update .github/workflows/format.yml Co-authored-by: Anghelo Carvajal <angheloalf95@gmail.com> * update matrix version list * finalize apply_fix_bss_patches.py --------- Co-authored-by: Anghelo Carvajal <angheloalf95@gmail.com>
2026-04-25Format extracted params: ObjSwitch (#2722)Dragorn421
* ObjSwitch params macros * format extracted ObjSwitch params
2026-04-17n64texconv: Improve automatic palette generation (#2731)Tharo
* n64texconv: Improve automatic palette generation * Reword documentation on automatic palettes
2026-04-12Texture software fixups (#2709)Dragorn421
* fixup n64texconv app usage text and check u32|u64 * fix memory leaks in build_from_png * yeet -Werror * Remove n64texconv/ap and create build_jfif * gitignore build_jfif binary, oops * Makefile: Add BUILD_FROM_PNG and BUILD_JFIF variables
2026-04-12`tools/assets` cleanup pass (mostly typing) (#2705)Dragorn421
* address easy todos * add type asserts * format * add type hints * more typing fixes * more asserts for typing * break long f string
2026-04-10Commit objects 2: `object_spot15_obj`, `object_spot16_obj`, ↵Dragorn421
`object_spot17_obj`, `object_syokudai`, `object_ta`, `object_timeblock` (#2645) * commit object_timeblock * commit object_ta * commit object_syokudai * commit object_spot17_obj * format * commit object_spot16_obj * fixup * commit object_spot15_obj
2026-04-07oot-gc-eu-dbg-2 OK (#2714)Dragorn421
* oot-gc-eu-dbg-proto OK * add gc-eu-dbg-proto to readme * mention overdump trimming for all debug roms * rename gc-eu-dbg-proto to gc-eu-dbg-2 Rationale: first version to make it into decomp (which should also be the first one that surfaces) gets an un-numbered version name, then the second version gets -2, and so on
2026-03-14tools/assets: fix build_from_png with Apple clang (#2724)Tanner Johnson
* tools/assets: fix build_from_png with Apple clang * tools/assets: use macro for build_from_png suffix limit
2026-03-10Audio: Fix note values (#2716)Tharo
Due to the layout of the pitch_frequencies lookup table, note values computed in extraction were reflected around middle C (midi note number 60). This didn't matter for matching, values would successfully roundtrip. However when using samples in external programs or converting soundfonts to standard formats the note values would lead to incorrect playback of sounds. This change corrects the note values so that external programs correctly infer the pitch of the sound when played at a particular MIDI key.
2026-03-09Allow `atblgen` to process `sequence_order.in` with empty lines in between ↵Anghelo Carvajal
lines for building on Macos (#2718) The present changes are a direct copy-paste from https://github.com/zeldaret/mm/pull/1850 This fixes building on macos due to a kinda specific issue with Apple clang. When trying to build on Macos (specifically MacOS 12, Monteray with Apple clang 13.0.0 (clang-1300.0.29.30), idk if other versions have this issue too) `make` stops with the following error from `atblgen`: ``` Failed to match line 1: "" regexec error: "regexec() failed to match" Error: Malformed build/n64-us/assets/audio/sequence_order.in? ``` `atblgen` makes the assumption that the `sequence_order.in` file has no extra data, spaces, empty lines, etc. but the file somehow ends up having empty lines between each line on macos. This file is created by using the C preprocessor to process `include/tables/sequence_table.h`. In normal circumstances this file should look like this snip, ``` (Sequence_0,NA_BGM_GENERAL_SFX) (Sequence_1,NA_BGM_AMBIENCE) (Sequence_2,NA_BGM_TERMINA_FIELD) (Sequence_3,NA_BGM_CHASE) (Sequence_4,NA_BGM_MAJORAS_THEME) ``` but it ends up looking like this instead ``` (Sequence_0,NA_BGM_GENERAL_SFX) (Sequence_1,NA_BGM_AMBIENCE) (Sequence_2,NA_BGM_TERMINA_FIELD) (Sequence_3,NA_BGM_CHASE) (Sequence_4,NA_BGM_MAJORAS_THEME) ``` which `atblgen` doesn't like. I believe this happens because there are lines with comments between each macro in [`sequence_table.h`](https://github.com/zeldaret/mm/blob/0877ce4adf28a8e73e05c3c58682273b8bf28749/include/tables/sequence_table.h) and for some reason this Apple clang version decided to preserve those empty lines The fix just makes `atblgen` skip empty lines. I threw `atblgen` to valgrind to check the fix was working as intended and noted a bunch of memory that was being free before exit, so I fixed them. I also noted the tools/audio makefile was not using `OPTFLAGS` when building those tools, so I fixed that too.
2026-03-02gc-eu-dbg OK (#2700)Dragorn421
* gc-eu-dbg OK * cleanup/fortify makefile and line_numbers.h changes * fill segments.csv * add gc-eu-dbg to Jenkinsfile * cleanup/fortify dmadata_table.h * add gc-eu-dbg to readme * fix checksum check and compressed checksum
2026-02-22Fix misc 23 (#2699)Dragorn421
* remove noop macros Fault_SetFontColor Fault_SetCharPad under PLATFORM_N64 * remove duplicate declarations in sfx.h * use render mode presets more * decimal for alpha values * fix some z_camera comments typos * fix references to preprocess.sh (used to be preprocess.py) * remove outdated comment on deps in makefile * code_800AD920 was renamed to z_viszbuf * fix capitalization typo: gOcarinaofTimeDesignTex -> gOcarinaOfTimeDesignTex * fix typo: gLinkAdultEyesClosedfTex, gLinkChildEyesClosedfTex -> remove extra f * fix typo: gCrstalSwitchRedTex, gCrstalSwitchBlueTex -> gCrystal * fix PARAMS_GET_U amount of bits
2026-02-22Assets system: format extracted actor params (and EnDoor params) (#2698)Dragorn421
* Assets system: format extracted actor params (and EnDoor params) * and actor list too... * and actor list required inclues too... * ENDOOR_PARAMS_GENERAL, ENDOOR_PARAMS -> ENDOOR_PARAMS, ENDOOR_PARAMS_NODATA
2026-02-17sampleconv: Improve WAV SMPL chunk handling, properly handle odd chunk sizes ↵Tharo
(#2689) * Read WAV SMPL chunks for note info when INST chunks are not available * Populate the SMPL chunk on conversion to wav * Properly handle unaligned (odd) WAV chunk sizes
2026-02-17Soundfont Compiler: Implement fine-tuning in the final tuning calculation ↵Tharo
(#2690)
2026-02-16Commit and split gameplay_keep (#2612)Tharo
* Commit and split gameplay_keep * Remove debugging bits * format * bss * merge gameplay_keep_0x400 into link_textures * Name some unused NTSC stuff * Rename files for some of the newly named unused NTSC stuff * name more gkeep files * format * spin_attack * name fish anims files * name some eff_ files * name a bunch more files * Add underscore in not-final names * cleanup limb enum names (remove g prefix) * split gameplay_keep_0x34020.c into gBugCrawlAnim.c + bug_skel.c * door.h -> door_skel.h * gArrowNAnim -> gArrowN_Anim * split arrow_anims into gArrow1_Anim + gArrow2_Anim * fix merge * bss * bss * Fix bss --------- Co-authored-by: Dragorn421 <Dragorn421@users.noreply.github.com>
2026-02-15Fix wrong BGM entry names (#2671)Jordan Longstaff
2026-01-27Commit code assets (#2691)Pepe20129
* Commit code assets * Have different ifdefs * Move to code files * Missed an include * Prevent code assets from having their source unnecessarily written
2025-07-15Add `ENV_LIGHT_SETTINGS_BLEND_RATE_AND_FOG_NEAR_PACK`, fix extracted ↵Dragorn421
negative `blendRate` (#2653) * Add ENV_LIGHT_SETTINGS_BLEND_RATE_AND_FOG_NEAR_PACK, fix extracted negative blendRate * -> BLEND_RATE_AND_FOG_NEAR
2025-07-15Finish rename scene lists (#2652)Dragorn421
2025-06-25Commit objects 1: `tp`, `triforce_spot`, `umajump`, `vase`, `yukabyun`, `zg` ↵Dragorn421
(#2634) * commit object_zg * commit object_yukabyun * commit object_vase * commit object_umajump * commit object_triforce_spot * commit object_tp * fixup1 * format
2025-06-10`object_zo`: finalize, split and commit (#2624)Dragorn421
2025-06-09Commit `object_ganon_anime3` (#2611)Dragorn421
* Commit object_ganon_anime3 * rm from write_source.txt * bss
2025-06-09Commit some `assets/overlays/` 2 (#2610)Dragorn421
* Commit some `assets/overlays/` 2 * format
2025-06-07Commit `do_action_static` (#2597)Dragorn421
* Commit `do_action_static` * fix * rm do_action_static from write_source.txt
2025-06-05Fix too long filenames for windows (#2591)Dragorn421
2025-06-05Better automatic names for Limbs and JointKeys, and implement more ↵Dragorn421
get_as_xml (#2590)
2025-06-04Remove "z64" prefix from all headers (#2518)fig02
* z64 - a * z64 - b * z64 - c * z64 - d * z64 - e * z64 - f * z64 - g * z64 - h * z64 - i * z64 - l * z64 - m * z64 - o * z64 - p * z64 - q * z64 - r * z64 - s * z64 - t * z64 - v * restore file * fix merge * fix merge --------- Co-authored-by: Dragorn421 <Dragorn421@users.noreply.github.com>
2025-06-04Commit some assets/overlays/ (#2582)Dragorn421
* Commit (inline) some assets/overlays/ * rm padding * ovl_file_choose
2025-06-03Commit `g_pn_*.c` files (#2573)Dragorn421
2025-06-01csdis.py: write `CS_CMD_ACTOR_CUE_` names in actor cue list header (#2561)Dragorn421
2025-05-31Commit `icon_item_24_static`, `icon_item_{fra,ger,jpn,nes}_static`, ↵Dragorn421
`item_name_static` (#2562) * commit icon_item_24_static * commit icon_item_LANG_static for LANG in fra,ger,jpn,nes * commit item_name_static
2025-05-27[assets system] Fix fusing resource buffer markers (#2548)Dragorn421
2025-05-26[assets] Better automatic names for subdata of Anim and Col data (#2543)Dragorn421
* better auto names for anim subdata * better auto names for col subdata
2025-05-26Commit `object_link_child` (#2542)Dragorn421
* add Vtx resources to object_link_child.xml * object_link_child.xml limb enum and tlut blob * cp extracted/... assets/... * start cleanup, delete useless declarations * inhibit object_link_child.c source gen * move textures _WIDTH and _HEIGHT to .c, remove useless tex decls * comments * fixup object_link_boy comments * remove unreferenced DLs decls * rm 0 padding * format the .c * fixup * bss
2025-05-25Force skybox split palettes to always contain 128 colors (#2535)Tharo
* Force skybox split palettes to always contain 128 colors, padding with 0 if necessary * Review
2025-05-25Finalize and commit `object_link_boy` (#2528)Dragorn421
* Finalize and commit object_link_boy * bss * format * dont write extracted/.../object_link_boy.c * sizes for dls and texs * fix OK * TEX_LEN
2025-05-25Introduce TEX_LEN macro for texture arrays lengths (#2541)Dragorn421
2025-05-23[assets] Explicit size for textures and dlists (#2534)Dragorn421
* Add `conf.EXPLICIT_SIZES = True` for making some array resources declare and define with explicit length * EXPLICIT_DL_AND_TEX_SIZES = True * revert rm line
2025-05-20Assets system utilities (#2529)Dragorn421
2025-05-20Add write_source.txt: list of files to write source for during assets ↵Dragorn421
extraction (#2531)
2025-05-18New assets system (#2481)Dragorn421
* wip: New assets system tm Builds gc-eu-mq-dbg OK from clean after 1) make setup 2) python3 -m tools.assets.extract -j 3) replace 0x80A8E610 with sShadowTex in extracted/gc-eu-mq-dbg/assets/overlays/ovl_En_Jsjutan/sShadowMaterialDL.inc.c 4) make various symbols in extracted data like sTex static * use variables from config.yml for gMtxClear and sShadowTex addresses * Write source with static for overlays using `HACK_IS_STATIC_ON` hack * gc-eu-mq-dbg OK from clean with `make setup && make` * implement more skeleton-related types, cleanups, fixups * fix extracted data to no longer produce compilation warnings * implement more of RoomShapeImage types * yeet XmlPath from ExternalFile usage * Implement PlayerAnimationDataResource (link_animetion data) * fix csdis CS_TIME extra arg * dmadata file names no longer hardcoded for gc-eu-mq-dbg * ntsc-1.0 OK * xml fixes * slightly improve standard output * rm extract_assets.py * generate and use Limb enums (TODO: check Skin skels and implement for Curve skels) * handle dependencies between xmls * introduce RawPointers xml attribute to ignore specific pointers and keep them raw * add tools/extract_assets.sh * fixups * only extract if xmls changed or if -f (force) is used * fixups, gc-eu OK * all versions OK * check attributes of xml resources elements * Implement legacy skelanime resources * fix ASSET_FILES_BIN_EXTRACTED/COMMITTED: look for .u8.bin specifically instead of just .bin * implement JFIFResource * fix png/jpg wildcards: look specifically for .u64.png .u32.png .u64.jpg * Makefile: Add rules to build .png, .bin and .jpg in assets/ too * start writing actual docs * extract sTransCircleDL and sTransWipeDL * misc cleanup/fixes, pygfxd 1.0.3 * refactor CDataExt.set_write callback args to use a dataclass * Move {} to in-source * misc * more progress on spec * fix missing braces in n64dd_error_textures.c * finish xml spec doc * assets xmls fixes * some cleanup, use `gNameTex_WIDTH/HEIGHT` macros in dlists * handle hackmode_syotes_room, fix compile * C build_from_png * rm tools/assets/bin2c * rm ZAPD * format * remove rule to generate dmadata_table.py * CC0 license (and some import cleanup) * dont try to build zapd (rmd) * simplify palettes with single user (ci images with a non-shared palette) * add docs on how images are handled * bss * allow -j N * fix n64texconv python bindings memory management * move -j at the end of calling extraction script * with -j, update last_extracts.json as each job completes rather than only if all complete * make interrupting less jank by making child processes ignore sigint * use enum names in `SCENE_CMD_SKYBOX_SETTINGS` * `multiprocessing.get_context("fork")` * import rich, except ImportError s * fix optional rich usage * .bss * .bss * .bss * assets extraction: -j -> -j$(N_THREADS) * .bss * change LIMB_NONE/MAX defaults to be FILE_OFFSET instead of SKELNAME * 0XHEX -> 0xHEX * fix bss * Proper includes for assets mostly proper, some includes like dlists resources always causing a sys_matrix.h include (when not every dlist references gIdentityMtx) could be done better * rm z64.h * rm z64.h take two * bss * Make .u64 suffix for pngs optional * fixup: rm .u64 suffix from n64dd image paths * Remove elemtype suffixes from .bin and .jpg files * Update images.md * some build_from_png cleanup, more error handling, comments * Handle skybox textures Introduce "sub-format" suffix for pngs, with sub-formats split_lo and split_hi being used for skybox textures * fixup for older python * improve collision output some * fully use SURFACETYPE[01] macros in writing extracted surface types * use WATERBOX_PROPERTIES in extracted waterboxes * some SceneCommandsResource cleanup * format EnvLightSettingsList output
2025-05-06Fix fix_bss.py (#2526)Dragorn421
generated by cgit v1.2.3 (git 2.25.1) at 2026-08-04 12:04:02 +0000