The chore/reorg branch drifted too far to merge cleanly. Captures the
worthwhile ideas as actionable proposals: clientctx removal, lazy packet
priority ordering, SessionStage interface, semaphoreIndex race fix, and
several medium/low-priority restructuring items.
Document client-side parser behaviour confirmed from mhfo-hd.dll:
- Container: 0x8000-byte per-chunk limit enforced by FUN_11525c60
- C0 metadata: client reads only m[0]–m[6]; m[7]–m[9] are not read,
so the constant-5 (m[8]) and the size-correlated m[9] are opaque to
the parser and can be set to any value
- C0 m[2]/m[5] role clarified: m[2]=0 is offset to str0 (no-op),
m[5]=str0_len is offset to str1 from strings section start
- C0 m[6] SceneRef stored as signed short; 0xFFFF = −1 for cat≠0
- C1 metadata: m[8]–m[17] are signed offsets — negative encodes
position from the post-0xFF dialog data section via ~value formula,
non-negative encodes position from the strings section start;
m[18–19] are read as individual bytes, not u16 pairs
Add TestScenarioRoundTrip_RealFiles covering 7 real scenario files
(cat=0/1/3, T101/T103, with and without chunk1). Tests skip gracefully
when game data is absent so CI stays green.
Decoded metadata structure from analysis of 145k+ real scenario files:
- Chunk0 m[0]/m[1] = CategoryID/MainID; m[5] = str0_len (offset to
str1); m[6] = MainID (cat=0) or 0xFFFF; m[8] = constant 5.
- Chunk1 m[9]=cumOff[2], m[10]=cumOff[1], m[14]=cumOff[3],
m[15]=total_string_bytes; m[11-13]/m[16-17] = dialog script offsets
beyond the 0xFF sentinel; m[20] = constant 5; m[21] ≈ data_size.
Update docs/scenario-format.md with full field tables for both chunks.
Closes#172. Scenario files in bin/scenarios/ can now be authored as
.json instead of .bin — the server compiles them to wire format on
load, falling back to .bin if no .json is present.
- Add ParseScenarioBinary / CompileScenarioJSON in scenario_json.go;
supports sub-header format (strings as UTF-8, metadata as base64),
inline format, and raw JKR blobs.
- Add PackSimple JKR type-3 (LZ77) compressor in jpk_compress.go,
ported from ReFrontier JPKEncodeLz.cs; round-trip tested against
UnpackSimple.
- Fix off-by-one in processDecode (jpk.go): last literal byte was
silently dropped for data that does not end on a back-reference.
- Wire loadScenarioBinary into handleMsgSysGetFile replacing the
inline os.ReadFile call; mirrors the existing loadQuestBinary pattern.
- Rewrite docs/scenario-format.md with full container/sub-header spec
and JSON schema examples.
Promote [Unreleased] to [9.3.0] - 2026-03-19 in CHANGELOG and mark
the bookshelf save pointer fix as Done in improvements.md (it was
applied post-RC1 but the doc still said Pending).
MSG_MHF_GET_EXTRA_INFO (0xA6) and MSG_MHF_GET_COG_INFO (0xC3) had
Parse() returning NOT IMPLEMENTED. The dispatch loop treats any Parse
error as a hard drop — no ACK is ever sent, so the client waits
indefinitely and effectively soft-locks when entering the G-rank
Workshop or Master Felyne (Cog) screens.
Fix: parse AckHandle (the only field we can confirm from the protocol)
and respond with doAckBufFail so the client receives a well-formed
buf-type ACK with error code 1. The client's fail branch for these
requests exits cleanly without reading response fields, avoiding the
read-past-EOF crash that an empty success ACK would cause.
The full response format for both packets is still unknown; a complete
implementation requires further RE. The TODO comments mark the gap.
Fixes#180.
Add // stub: unimplemented to 70 empty game-feature handlers and
// stub: reserved to 56 protocol-reserved slots in handlers_reserve.go,
making them discoverable via grep. Add docs/unimplemented.md listing all
unimplemented handlers grouped by subsystem with descriptions.
All tracked items in both files have been resolved or accepted.
The content is superseded by CLAUDE.md architecture notes and commit
history; retaining fully-struck-through docs adds noise without value.
RE'd putAdd_ud_point (FUN_114fd490) and putAdd_ud_tactics_point
(FUN_114fe9c0) from the ZZ client DLL via Ghidra decompilation.
MsgMhfAddUdPoint fields: QuestPoints (sum of 11 category accumulators
earned per quest) and BonusPoints (kiju prayer song multiplier extra).
MsgMhfAddUdTacticsPoint fields: QuestID and TacticsPoints.
Adds diva_points table (migration 0009) for per-character per-event
point tracking, with UPSERT-based atomic accumulation in the handler.
RE'd putDisplayed_achievement from ZZ client DLL via Ghidra: the packet
sends opcode + 1 zero byte with no achievement ID, acting as a blanket
"I saw everything" signal.
Server changes:
- Track per-character last-displayed levels in new displayed_levels
column (migration 0008)
- GetAchievement compares current vs displayed levels per entry
- DisplayedAchievement snapshots current levels to clear notifications
- Repo, service, mock, and 3 new service tests
Protbot changes:
- New --action achievement: fetches achievements, shows rank-up markers,
sends DISPLAYED_ACHIEVEMENT, re-fetches to verify notifications clear
- Packet builders for GET/ADD/DISPLAYED_ACHIEVEMENT
Load and validate rengoku_data.bin once during server initialization
instead of reading it from disk on every client request. The file is
static ECD-encrypted config data (~4.9 KB) that never changes at
runtime. Validation checks file size and ECD magic bytes, logging a
warning if the file is missing or invalid so misconfiguration is
caught before any client connects.
Lobby search now returns only quest-bound players (QuestReserved) instead
of all reserved slots, matching retail behavior. The new field is
pre-collected under server lock before stage iteration to respect
Server.Mutex → Stage.RWMutex lock ordering.
Replaced three TODOs with RE documentation from Ghidra decompilation of
mhfo-hd.dll ZZ:
- Log key off-by-one: putRecord_log/putTerminal_log pass size 0 for the
key field in ZZ, so the stored key is unused beyond issuance
- User search padding: ZZ per-entry parser confirms 40-byte block via
memcpy(dst, src+8, 0x28); G2 DLL analysis inconclusive (stripped)
- Player count: field at entry offset 0x08 maps to struct param_1[0xe]
Alliance applications were hardcoded to always-open. Add a `recruiting`
column to guild_alliances and handle OperateJoint actions 0x06 (Allow)
and 0x07 (Deny) confirmed via Wii U debug symbols. Only the parent
guild leader can toggle the setting, matching the existing disband guard.
Filed #164-#168 for all 8 remaining TODOs. Added Tracker column
with issue links, fixed stale line numbers, added MhfAddUdPoint
stub entry, and expanded the suggested execution order.
Add REST-idiomatic DELETE method as alias for POST .../delete.
Add 8 router-level tests exercising Bearer auth, invalid tokens,
soft delete, export body decoding, and MaxLauncherHR capping.
Create OpenAPI 3.1.0 specification covering all v2 endpoints.
The CLAUDE.md layered architecture section implied all handlers go
through services. In practice, services handle cross-repo coordination
while handlers call repos directly for simple CRUD. Updated the
diagram, added a services table, and added an "Adding Business Logic"
guide. Marked improvements.md item 4 as done.
Static data (GZ monster prices, LB prices, wanted list) cluttered the
handler with 130 lines of table literals. Moving them to a dedicated
tables file keeps the handler focused on serialization logic.
Enables isolated unit tests for tower, festa, rengoku, diva, event,
misc, mercenary, and cafe handlers. All 21 repo interfaces now have
mock implementations in repo_mocks_test.go.
mockGuildRepoForMail and mockGuildRepoOps each implemented different
subsets of the 68-method GuildRepo interface. Adding any new method
required updating both mocks. Merged into a single mockGuildRepo with
configurable struct fields for error injection and no-op defaults for
the rest.
KeyQuest set, Rights, and Teleport commands silently used zero values
when given malformed arguments (bad hex, non-integer coords). Now they
send the existing i18n error messages back to the player instead.
Several handler files discarded errors from repository and service
calls, creating nil-dereference risks and silent data corruption:
- guild_adventure: 3 GetByCharID calls could panic on nil guild
- gacha: GetGachaPoints silently returned zero balances on DB error
- house: HasApplication called before nil check on guild;
GetHouseContents error discarded with 7 return values
- distitem: 3 distRepo calls had no error logging
- guild_ops: Disband/Leave service errors were invisible
- shop: gacha type/weight/fpoint lookups had no error logging
- discord: bcrypt error could result in nil password being set
Codecov requires an account and token to function. Replace it with
a self-contained `go tool cover` step that fails the build if total
coverage drops below 50% (currently ~58%). This catches test
regressions without external service dependencies.
The 5 remaining handler files (seibattle, kouryou, scenario, distitem,
guild_mission) were already covered by commit 6c0269d but the doc
was not updated at the time.
Add 30 new tests to handlers_commands_test.go covering previously
untested paths: raviente with semaphore (start, multiplier, ZZ-only
sed/res commands, version gating), course enable/disable/locked/alias,
reload with other players and objects, help filtering for non-op vs op,
ban error paths and long-form duration aliases, and disabled-command
gating for all 12 commands. Total: 62 tests, all passing with -race.
The guild daily RP rollover TODO in handlers_guild_ops.go was stale —
the feature was already implemented via lazy rollover in
handlers_guild.go. Several other items in technical-debt.md were also
resolved in prior commits (typos, db guard investigation). Updated
the doc to reflect current state and bumped codecov-action to v5.
Add SJISToUTF8Lossy() that wraps SJISToUTF8() and logs decode errors at
slog.Debug level. Replace all 31 call sites across 17 files that previously
discarded the error with `_, _ =`. This makes garbled text from malformed
SJIS client data debuggable without adding noise at default log levels.
Replace db.Begin() with db.BeginTxx(context.Background(), nil) across all
8 remaining call sites in repo_guild.go, repo_guild_rp.go, repo_festa.go,
and repo_event.go. Use deferred Rollback() instead of explicit rollback
at each error return, eliminating 15 manual rollback calls.
Players could never claim monthly guild items because the handler
always returned 0x01 (claimed). Now tracks per-character per-type
(standard/HLC/EXC) claim timestamps in the stamps table, comparing
against the current month boundary to determine claim eligibility.
Adds MonthStart() to gametime, extends StampRepo with
GetMonthlyClaimed/SetMonthlyClaimed, and includes schema migration
31-monthly-items.sql.
fmt.Sprintf inside zap logger calls defeats structured logging,
making log aggregation and filtering harder. All 6 sites now use
proper zap fields (zap.Uint32, zap.Uint8, zap.String).
LoopDelay had no viper.SetDefault, so omitting it from config.json
caused a zero-value (0 ms) busy-loop in the recv loop. Default is
now 50 ms, matching config.example.json.
main.go always sets both Channels and Registry together, making the
Channels fallback paths dead code. This removes:
- Server.Channels field from the Server struct
- 3 if/else fallback blocks in handlers_session.go (replaced with
Registry.FindChannelForStage, SearchSessions, SearchStages)
- 1 if/else fallback block in handlers_guild_ops.go (replaced with
Registry.NotifyMailToCharID)
- 3 method fallbacks in sys_channel_server.go (WorldcastMHF,
FindSessionByCharID, DisconnectUser now delegate directly)
Updates anti-patterns.md #6 to "accepted design" — Session struct is
appropriate for this game server's handler pattern, and cross-channel
coupling is now fully routed through the ChannelRegistry interface.
All guild subsystem tables are now migrated into repo_guild.go.
21 repository files cover all major subsystems. Only 5 inline SQL
queries remain across 3 handler files. Updated summary table and
refactoring priority list to mark completed items.
Per anti-patterns.md item #9, guild-related SQL was scattered across
~15 handler files with no repository abstraction. Following the same
pattern established by CharacterRepository, this centralizes all
guilds, guild_characters, and guild_applications table access into a
single GuildRepository (~30 methods).
guild_model.go and handlers_guild_member.go are trimmed to types and
pure business logic only. All handler files (guild_*, festa, mail,
house, mercenary, rengoku) now call s.server.guildRepo methods
instead of direct DB queries or methods on domain objects.
Add 18 new typed methods to CharacterRepository (ReadTime, SaveTime,
SaveInt, SaveBool, SaveString, ReadBool, ReadString, LoadColumnWithDefault,
SetDeleted, UpdateDailyCafe, ResetDailyQuests, ReadEtcPoints, ResetCafeTime,
UpdateGuildPostChecked, ReadGuildPostChecked, SaveMercenary, UpdateGCPAndPact,
FindByRastaID) and migrate ~56 inline SQL queries across 13 handler files.
Pure refactor — zero behavior change. Each handler produces identical SQL
with identical parameters. Cross-table JOINs and bulk CharacterSaveData
operations are intentionally left out of scope.
Centralizes all characters table SQL behind a CharacterRepository struct
in repo_character.go. The 4 existing helpers (loadCharacterData,
saveCharacterData, readCharacterInt, adjustCharacterInt) now delegate to
the repository, keeping identical signatures so all ~70 callsites remain
unchanged. Direct queries in handlers_session.go, sys_channel_server.go
(DisconnectUser), and handlers_mail.go are also migrated.
Pure refactor with zero behavior change — first step toward eliminating
the ~130 scattered character queries identified in anti-patterns #9.
Add readCharacterInt/adjustCharacterInt helpers for single-column
integer operations on the characters table. Eliminates fmt.Sprintf
SQL construction in handlers_misc.go and replaces inline queries
across cafe, kouryou, and mercenary handlers.
Second round of protocol constant extraction: adds constants_time.go
(secsPerDay, secsPerWeek), constants_raviente.go (register IDs,
semaphore constants), and named constants across 14 handler files
replacing raw hex/numeric literals. Updates anti-patterns doc to
mark #4 (magic numbers) as substantially fixed.
Binary I/O (#5): all 12 remaining encoding/binary calls are
legitimate (zero-alloc spot-reads, random-access into game blobs).
Copy-paste handlers (#8): loadCharacterData/saveCharacterData helpers
now cover standard blob patterns.
Also upgrades saveCharacterData to send doAckSimpleFail on oversize
payloads and DB errors, and migrates handleMsgMhfSaveScenarioData
to the improved helper.
handleMsgSysOperateRegister returned without sending an ACK when the
payload was empty, causing the client to softlock waiting for a response.
Send doAckBufSucceed with nil data on the early-return path to match
the success-path ACK type.
Also update tests to expect error returns instead of panics from
unimplemented Build/Parse stubs (matching prior panic→error refactor),
and mark resolved anti-patterns in docs.
Covers 13 identified anti-patterns across error handling, architecture,
concurrency, and code organization with severity ratings and
recommended refactoring priorities.
