Shutdown now proceeds in three phases: close listeners immediately on
signal, broadcast the in-game countdown, wait up to ShutdownDrainSeconds
(default 30) for sessions to disconnect naturally via DrainPassive, then
force-close any stragglers. This prevents players from entering new
quests after the countdown starts, and lets mid-quest sessions finish
saving without being killed mid-write. A second SIGINT during passive
drain cancels it so the force-close phase runs immediately.
Renamed DisableSoftCrash -> DisableShutdownCountdown since the flag
controls the countdown, not crash behaviour. Existing config.json files
keep working via a Viper alias on the legacy key.
Closes#179.
Brings 53 develop commits (i18n, Diva, campaign, guild invites, save
transfer, return/rookie guilds, hunting tournament, JSON quest/scenario
loaders, Ghidra-derived user binary parsing, and misc fixes) onto main
now that 9.3.2 has been tagged and released.
Resolves two overlap zones:
1. Migration number collision. Main shipped 0010_fix_zero_rasta_id and
0011_fix_stale_boost_time in 9.3.2; develop had independently
numbered 0010_campaign..0015_tournament. The migration runner keys
applied versions by integer, so coexisting files with the same
numeric prefix would silently skip each other. Develop's files have
been renumbered to 0016..0021, leaving main's 0010/0011 intact. A
schema_version rename script is required on any server that had
already applied the old develop numbers (only frontier.mogapedia.fr
at the time of this merge).
2. CHANGELOG.md. Develop's in-progress feature entries move into
[Unreleased] with updated migration references; the [9.3.2] section
is preserved verbatim.
main.go version string bumped to 9.4.0-dev to mark the new cycle.
Full test suite (go test -race ./...) passes.
Live-server testing via protbot surfaced an inconsistency between
GetBoostTimeLimit and GetBoostRight: on the same character, the
former reported a far-future boost limit (2288912640 = year 2042)
while the latter correctly reported "expired / available". The two
handlers read the same boost_time row and disagreed.
Root cause: GetBoostTimeLimit was doing a naked uint32(int64) cast
on boostLimit.Unix(). The test character's boost_time was actually
year 1906 (a pre-1970 sentinel left behind by the pre-#187 bug),
whose negative int64 Unix timestamp wraps through uint32 to a huge
positive value the client interprets as a permanently active boost.
Harmonise the guard with GetBoostRight: return 0 whenever the stored
boost_time is not strictly after TimeAdjusted(), covering both the
pre-1970 wraparound and the "already expired" case.
Add a healing migration (0011_fix_stale_boost_time) that NULLs out
any boost_time column older than 1970 or more than 10 years in the
future, so affected characters recover on upgrade without waiting
for a fresh boost start.
Regression test uses the exact year-1906 value observed on the live
frontier.mogapedia.fr test account.
Both bugs surfaced when running protbot against a live Erupe instance.
LoadColumnWithDefault now treats an empty bytea ('\x', len 0) the same
as NULL. The postgres driver returns a non-nil empty slice for empty
bytea, so the prior `data == nil` check let a zero-byte slice reach
handleMsgMhfReceiveGachaItem, which forwarded it to the client as a
malformed gacha_items response. The MHF client interprets a zero-byte
count field as a protocol error and crashes the gacha menu, matching
the #175 symptom class. The handler also gains a defensive fallback
for len(data) == 0 in case another caller hits the same edge.
handleMsgMhfGetBoostTimeLimit was sending two ACKs for a single
request: doAckBufSucceed with the real payload, then an unconditional
doAckSimpleSucceed on the same ack handle. The second ACK was dead on
arrival (the handle was already consumed) but is a latent protocol
bug. Drop it and update the regression test that was asserting the
buggy 2-packet behavior.
Previously GetItemsForEntry/GetGuaranteedItems silently swallowed
StructScan errors, so misconfigured rows (item_type > 255 or
item_id/quantity > 65535) disappeared from rewards with no trace,
making config bugs hard to diagnose without a DB dump.
Pass a zap.Logger into GachaRepository and emit a Warn pointing at
the likely cause and the offending entry/gacha ID.
The pre-106cf85 SaveMercenary bug could overwrite a character's NULL
rasta_id with 0, a value not drawn from rasta_id_seq that causes silent
save failures. The code fix prevents new occurrences but leaves already
affected rows stuck. Migration 0010 sets rasta_id=NULL where it is 0 so
existing users auto-heal on upgrade.
Two bugs in handleMsgMhfEnumerateQuest affecting reward multipliers:
1. A Value > 0 filter silently dropped any multiplier set to exactly
0.0 in config, causing the client to fall back to its hardcoded
default (100%). So ZennyMultiplier: 0.0 produced *full* zenny
instead of none. Removed the filter so zero values are sent
verbatim.
2. uint16(float32(0.20) * 100) yields 19, not 20, due to float32
representation of 0.20 being ~0.19999998. Added a
multiplierToTuneValue helper using math.Round and applied it to
all 18 multiplier call sites (HRP/SRP/GRP/GSRP/Zenny/GZenny/
Material/GMaterial/GCP/GUrgent and NC variants).
Three focused tests asserting the response payloads when the config flags
are set: GetBoostTimeLimit and GetBoostRight return zero when
DisableBoostTime is true, and UseKeepLoginBoost returns an all-zero ack
when DisableLoginBoost is true. Verified to fail against the pre-fix
source.
GetBoostTimeLimit and GetBoostRight now respect DisableBoostTime, and
UseKeepLoginBoost now respects DisableLoginBoost. Also fix a latent
zero-time.Time wraparound in GetBoostTimeLimit that caused the
"Boost Time" overlay to appear on fresh characters regardless of
config, since time.Time{}.Unix() cast to uint32 yields a large value
the client interprets as an active timestamp.
Reverse-engineered from mhfo-hd.dll via Ghidra: type 1 = character name,
type 2 = player profile (208B), type 3 = equipment snapshot (384B).
Adds structured zap logging and size validation warnings to
handleMsgSysSetUserBinary.
Backports two softlock fixes and playtime regression fix from main.
Also removes handleMsgMhfEnterTournamentQuest from the nil-stub test
list — it's a real DB-backed handler and has its own dedicated test.
MSG_CA_EXCHANGE_ITEM and MSG_MHF_USE_UD_SHOP_COIN had Parse() returning
"NOT IMPLEMENTED". The dispatch loop in handlePacketGroup treats any
Parse error as a silent drop — no ACK is sent, causing the client to
wait indefinitely (softlock). Reported on 9.3.0-rc1 for forge item
purchases and Hunting Road N-point interactions.
Fix follows the pattern from d27da5e: parse only the AckHandle, return
nil from Parse, and respond with doAckBufFail so the client's error
branch exits cleanly without reading response fields.
updateSaveDataWithStruct only wrote RP and KQF into the blob, leaving
the playtime field stale. On each reconnect, GetCharacterSaveData read
the old in-game counter from the blob and reset s.playtime to it,
rolling back all progress accumulated during the previous session.
Playtime is now persisted into the blob alongside RP, using the same
S6 mode guard as the read path in updateStructWithSaveData.
Exposes the server's configured client mode in a form that mhf-outpost
and other launcher tools can consume to warn users when their local game
version does not match what the server expects.
Returns clientMode (raw, e.g. "G9.1") and manifestId (normalized for
mhf-outpost: lowercase, dots stripped, e.g. "g91"). No auth required.
- alliance recruiting default: change DEFAULT true → false in both
0001_init.sql and 0004_alliance_recruiting.sql; new alliances should
not be open for recruitment by default (TestSetAllianceRecruiting)
- guild_invites migration: add IF NOT EXISTS so re-running migrations
on an existing DB does not fail with "relation already exists"
(TestMigrateExistingDBWithoutSchemaVersion)
- test character name: shorten "Idem_Rollover_Leader" (19 chars) to
"IdemRollLeader" to fit the VARCHAR(15) constraint
(TestRolloverDailyRP_Idempotent)
applyMigration() already wraps each file in a db.Begin()/tx.Commit()
transaction. The inner BEGIN/COMMIT in these files caused the outer
transaction to commit early, leaving applyMigration trying to insert
into schema_version on an idle connection:
pq: unexpected transaction status idle
Affected: 0012_guild_invites, 0014_return_guilds, 0015_tournament.
The constant was declared and documented but never used, causing a
golangci-lint failure. Wire it into compileScenario() so oversized
chunks are rejected with a clear error rather than silently served
to the client (which discards them per FUN_11525c60 RE findings).
Resolves CHANGELOG.md conflict: preserve develop's [Unreleased] block,
insert the [9.3.1] section from main, remove the duplicate
DisableSaveIntegrityCheck entry that had been in [Unreleased].
LoopDelay is 0 in test servers (bare struct, no Viper defaults), so
sendLoop spins without delay and satisfies conditions in <1ms. The
10ms poll interval was the new bottleneck — dropping to 1ms cuts the
channelserver test suite from ~136s to ~5s.
Blind sleeps accumulate serially (no t.Parallel anywhere) and inflate
under the race detector's scheduling overhead — contributing to the
~136s channelserver test run time.
Replace ~75 arbitrary sleeps (50ms–1s) across 7 test files with 2s
polling loops that exit as soon as the expected condition holds. Sleeps
that are genuinely intentional (race-condition stress tests, cache
expiry, temporal spacing in timestamp tests, backpressure pacing) are
left untouched.
The fixed 100ms sleep was too short for sendLoop goroutines to drain
under the race detector's scheduling overhead, causing intermittent
count=4/want=5 failures. Replace with a 2s polling loop that exits
as soon as all sessions report delivery.
Admins importing saves from another server instance will hit the
hash mismatch error with no guidance. The log message now tells them
to set DisableSaveIntegrityCheck=true or null the hash for the
specific character via SQL.
TestLangCompleteness uses reflection to walk the i18n struct and fail
on any empty string field, catching incomplete translations at CI time.
Running it immediately found three missing strings in lang_jp.go
(playtime, timer.enabled, timer.disabled), which are now filled in.
CHANGELOG updated with i18n refactor, FR/ES languages, and the new test.
sys_language.go now contains only the i18n struct, Bead type, lookup
helpers, and a minimal getLangStrings dispatcher. All string data lives
in lang_en.go and lang_jp.go, making it straightforward to add a new
language by creating a single self-contained file.
Wire format for MsgMhfEnterTournamentQuest (0x00D2) derived from
mhfo-hd.dll binary analysis (FUN_114f4280). Five new tables back
the full lifecycle: schedule, cups, sub-events, player registrations,
and run submissions. All six tournament handlers are now DB-driven:
- EnumerateRanking: returns active tournament schedule with cups and
sub-events; computes phase state byte from timestamps
- EnumerateOrder: returns per-event leaderboard ranked by submission
time, with SJIS-encoded character and guild names
- InfoTournament: exposes tournament detail and player registration
state across all three query types
- EntryTournament: registers player and returns entry handle used by
the client in the subsequent EnterTournamentQuest packet
- EnterTournamentQuest: parses the previously-unimplemented packet and
records the run in tournament_results
- AcquireTournament: stubs rewards (item IDs not yet reversed)
Seed data (TournamentDefaults.sql) reproduces tournament #150 cups and
sub-events so a fresh install has a working tournament immediately.
New/returning players are now auto-assigned to temporary holding guilds
on MSG_MHF_ENTRY_ROOKIE_GUILD (pkt.Unk=0 → rookie guild, ≥1 → comeback
guild). Guilds are created on demand and capped at 60 members. Players
leave via the OperateGuildGraduateRookie/Return actions. The guild info
response now reports isReturnGuild from the DB instead of hardcoded false.
Migration 0014_return_guilds adds return_type to the guilds table.
Adds two complementary paths for transferring character save data between
Erupe instances without breaking the SHA-256 integrity check system:
- `cmd/saveutil/`: admin CLI with `import`, `export`, `grant-import`, and
`revoke-import` subcommands. Direct DB access; no server running required.
- `POST /v2/characters/{id}/import`: player-facing API endpoint gated behind
a one-time token issued by `saveutil grant-import` (default TTL 24 h).
Token is validated and consumed atomically to prevent TOCTOU races.
- Migration `0013_save_transfer`: `savedata_import_token` and
`savedata_import_token_expiry` columns on `characters` table.
- Both paths decompress incoming savedata and recompute the SHA-256 hash
server-side, so the integrity check remains valid after import.
- README documents both methods and the per-character hash-reset workaround.
Closes#183.
The SHA-256 integrity check introduced in migration 0007 blocks saves
when a character's savedata blob is imported from another server instance,
because the stored hash in the target DB no longer matches the new blob.
Adding DisableSaveIntegrityCheck (default: false) lets server operators
bypass the check to unblock cross-server save transfers. A warning is
logged each time the check is skipped so the flag's use is auditable.
Documents the per-character SQL alternative in CHANGELOG:
UPDATE characters SET savedata_hash = NULL WHERE id = <id>
Closes#183.
Scout invitations were stored in guild_applications with type 'invited',
forcing the scout list response to use charID as the invitation ID — a
known hack that made CancelGuildScout semantically incorrect.
Introduce a dedicated guild_invites table (migration 0012) with a serial
PK. The scout list now returns real invite IDs and actual InvitedAt
timestamps. CancelGuildScout cancels by PK. AcceptInvite and DeclineInvite
operate on guild_invites while player-applied applications remain in
guild_applications unchanged.
SaveColumn and SaveMercenary now check RowsAffected() and return a
wrapped ErrCharacterNotFound when 0 rows are updated, preventing
silent data loss when a character ID is missing or mismatched.
AdjustInt already detects this via its RETURNING scan — no change.
Daily mission packet structs (Get/SetDailyMission*) now parse the
AckHandle instead of returning NOT IMPLEMENTED, letting handlers
send empty-list success ACKs and avoiding client softlocks.
Also adds tests for dashboard stats endpoint and for five guild
repo methods (SetAllianceRecruiting, RolloverDailyRP,
AddWeeklyBonusUsers, InsertKillLog, ClearTreasureHunt) that
had no coverage.
Add ShutdownAndDrain to the channel server (issue #179 non-breaking
subset): on SIGTERM/SIGINT, force-close all active sessions so that
logoutPlayer runs for each one (saves character data, cleans up stages
and semaphores), then poll until the session map empties or a 30-second
context deadline passes. Existing Shutdown() is unchanged.
Add ShutdownCountdownSeconds int config field (default 10) alongside
DisableSoftCrash so operators can tune the broadcast countdown without
patching code. A zero value falls back to 10 for safety.
Fix pre-existing test failures: MsgMhfAddRewardSongCount has a complete
Parse() implementation so it no longer belongs in the "NOT IMPLEMENTED"
parse test list; its handler test is updated to pass a real packet and
assert an ACK response instead of calling with nil.
JSON-authored quests were serveable via MSG_SYS_GET_FILE but invisible
on the event quest board because loadQuestFile only read .bin files.
CompileQuestJSON already produces the uncompressed binary layout that
the event-board parser expects, so no decompression step is needed.
Also update rengoku priority tests to match the .bin-first convention
fixed in the previous commit.
All three binary loaders (quests, scenarios, rengoku) now follow the same
priority: .bin first, .json as fallback. The previous reversed order
(.json first) was confusing for operators with both files present.
Add jkrMagic and scenarioChunkSizeLimit named constants, and expand
comments throughout scenario_json.go to reflect confirmed client
behaviour: the 0x8000-byte per-chunk limit, which C0 metadata fields
the parser actually reads (m[0]–m[6] only), the signed-offset
encoding used for C1 m[8]–m[17], and per-byte roles in the 8-byte
sub-header. Inline magic literal replaced with the named constant.
Add TestScenarioRoundTrip_RealFiles covering 7 real scenario files
(cat=0/1/3, T101/T103, with and without chunk1). Tests skip gracefully
when game data is absent so CI stays green.
Decoded metadata structure from analysis of 145k+ real scenario files:
- Chunk0 m[0]/m[1] = CategoryID/MainID; m[5] = str0_len (offset to
str1); m[6] = MainID (cat=0) or 0xFFFF; m[8] = constant 5.
- Chunk1 m[9]=cumOff[2], m[10]=cumOff[1], m[14]=cumOff[3],
m[15]=total_string_bytes; m[11-13]/m[16-17] = dialog script offsets
beyond the 0xFF sentinel; m[20] = constant 5; m[21] ≈ data_size.
Update docs/scenario-format.md with full field tables for both chunks.
Closes#172. Scenario files in bin/scenarios/ can now be authored as
.json instead of .bin — the server compiles them to wire format on
load, falling back to .bin if no .json is present.
- Add ParseScenarioBinary / CompileScenarioJSON in scenario_json.go;
supports sub-header format (strings as UTF-8, metadata as base64),
inline format, and raw JKR blobs.
- Add PackSimple JKR type-3 (LZ77) compressor in jpk_compress.go,
ported from ReFrontier JPKEncodeLz.cs; round-trip tested against
UnpackSimple.
- Fix off-by-one in processDecode (jpk.go): last literal byte was
silently dropped for data that does not end on a back-reference.
- Wire loadScenarioBinary into handleMsgSysGetFile replacing the
inline os.ReadFile call; mirrors the existing loadQuestBinary pattern.
- Rewrite docs/scenario-format.md with full container/sub-header spec
and JSON schema examples.
nullcomp's passthrough path returns non-cmp-header data as-is without
error, which is correct for old uncompressed saves. However, a corrupt
backup slot containing garbage shorter than the minimum save layout
(100 bytes) would pass Decompress() and then panic in
updateStructWithSaveData() with a slice-bounds error at the name field
read (offset 88–100).
Add a minSaveSize check after each backup decompresses; skip the slot
if the result is too small. Also document the campaign system and the
fix in CHANGELOG under Unreleased.
Migration runner wraps each file in its own transaction via db.Begin().
Having an explicit BEGIN/END inside caused 'unexpected transaction status idle'.
Other migrations use no transaction wrapper.
Operators can now define Hunting Road configuration in a plain JSON file
(rengoku_data.json) instead of maintaining an opaque pre-encrypted binary.
The JSON is parsed, validated, assembled into the binary layout, and
ECD-encrypted at startup; rengoku_data.bin is still used as a fallback.
JSON schema covers both road modes (multi/solo) with typed floor and
spawn-table entries — floor number, spawn-table index, point multipliers,
and per-slot monster ID/variant/weighting fields. Out-of-range references
are caught at load time before any bytes are written.
Port ECD encryption/decryption from ReFrontier (C#) and FrontierTextHandler
(Python) into common/decryption. The cipher uses a 32-bit LCG key stream with
an 8-round Feistel-like nibble transformation and CFB chaining; all six key
sets are supported, key 4 being the default for all MHF files.
On startup, loadRengokuBinary now decrypts (ECD) and decompresses (JKR) the
binary to validate pointer bounds and entry counts, then logs a structured
summary (floor counts, spawn table counts, unique monster IDs). Failures are
non-fatal — the encrypted blob is still cached and served to clients unchanged,
preserving existing behaviour. Closes#173.
When primary savedata fails its SHA-256 integrity check, query
savedata_backups in recency order and return the first slot that
decompresses cleanly. Recovery is read-only — the next successful
Save() overwrites the primary with fresh data and a new hash,
self-healing the corruption transparently.
Closes#178
Adds human-readable JSON as an alternative quest format for bin/quests/.
The server tries .bin first (full backward compatibility), then falls back
to .json and compiles it to the MHF binary wire format on the fly.
JSON quests cover all documented fields: text (UTF-8 → Shift-JIS),
objectives (all 12 types), monster spawns, reward tables, supply box,
stages, rank requirements, variant flags, and forced equipment.
Also adds ParseQuestBinary for the reverse direction, enabling tools to
round-trip quests and verify that JSON-compiled output is bit-for-bit
identical to a hand-authored .bin for the same quest data.
49 tests: compiler, parser, 13 round-trip scenarios, and golden byte-
level assertions covering every section of the binary layout.
