Merge unstable into development (#2173)

* Remove more scene synchronized

* Fix worktop options not appearing

* Format code [skip actions]

* Fix delay with server tasks

* Format code [skip actions]

* Fully fix fairy clock (#2146)

* Fix scene transition

* fully fix fairy clock

* Re-add call to `Player#updatePlayerGameTime`

* Format code [skip actions]

* Initialize the script loader in `ResourceLoader#loadAll`

* Fix region removal checking

* Format code [skip actions]

* Use Lombok's `EqualsAndHashCode` for comparing scene regions

* Format code [skip actions]

* Move 'invalid gather object' to `trace`

* Add more information to the 'unknown condition handler' message

* Move invalid ability action to trace

* Make `KcpTunnel` public

* Validate the NPC being talked to

* Format code [skip actions]

* NPCs are not spawned server side; change logic to handle it

* Format code [skip actions]

* unload scene when there are no players (#2147)

* unload scene when there are no players

* Update src/main/java/emu/grasscutter/game/world/Scene.java

Co-authored-by: Magix <27646710+KingRainbow44@users.noreply.github.com>

---------

Co-authored-by: Magix <27646710+KingRainbow44@users.noreply.github.com>

* Check if a command should be copied or HTTP should be used

* Lint Code [skip actions]

* Fix character names rendering incorrectly

* Add basic troubleshooting command

* Implement handbook teleporting

also a few formatting changes and sort data by logical sense

* Fix listener `ConcurrentModificationException` issue

* Add color change to `Join the Community!`

* Lint Code [skip actions]

* Make clickable buttons appear clickable

* Remove 'Mechanicus' entities from the list of entities

* Format code [skip actions]

* Fix going back returning a blank screen

* Implement entity spawning

* Add setting level to entity card

* Add support for 'plain text' mode

* Make descriptions of objects scrollable

* Lint Code [skip actions]

* Format code [skip actions]

* Change the way existing hooks work

* Format code [skip actions]

* Upgrade Javalin to 5.5.0 & Fix project warnings

* Upgrade logging libraries

* Fix gacha mappings static file issue

* Add temporary backwards compatability for `ServerHelper`

* Format code [skip actions]

* Remove artifact signatures from VCS

* Fix forge queue data protocol definition

* Run `spotlessApply`

* Format code [skip actions]

* Download data required for building artifacts

* Add call for Facebook logins

* Add the wiki page as a submodule

* Format code [skip actions]

* Update translation (#2150)

* Update translation

* Update translation

* Separate the dispatch and game servers (pt. 1)

gacha is still broken, handbook still needs to be done

* Format code [skip actions]

* Separate the dispatch and game servers (pt. 2)

this commit fixes the gacha page

* Add description for '/troubleshoot'

* Set default avatar talent level to 10

* Separate the dispatch and game servers (pt. 3)

implement handbook across servers!

* Format code [skip actions]

* Update GitHub Actions to use 'download-file' over 'wget'

* Gm handbook lmao (#2149)

* Fix font issue

* Fix avatars

* Fix text overflow in commands

* Fix virtualized lists and items page 😭😭

* magix why 💀

* use hover style in all minicards

* button

* remove console.log

* lint

* Add icons

* magix asked

* Fix overflow padding issue

* Fix achievement text overflow

* remove icons from repo

* Change command icon

* Add the wiki page as a submodule

* total magix moment

* fix text overflow in commands

* Fix discord button

* Make text scale on Minicard

* import icons and font from another source

* Add hover effects to siebar buttons

* move font and readme to submodule repo

* Make data folder a submodule

* import icons and font from data submodule

* Update README.md

* total magix moment

* magix moment v2

* submodule change

* Import `.webp` files

* Resize `HomeButton`

* Fix 'Copy Command' reappearing after changing pages

---------

Co-authored-by: KingRainbow44 <kobedo11@gmail.com>

* Lint Code [skip actions]

* Download data for the build, not for the lint

* format imports

this is really just to see if build handbook works kek

* Implement proper handbook authentication (pt. 1)

* Implement proper handbook authentication (pt. 2)

* Format code [skip actions]

* Add quest data dumping for the handbook

* Change colors to fit _something suitable_

* Format code [skip actions]

* Fix force pushing to branches after linting

* Fix logic of `SetPlayerPropReq`

* Move more group loading to `trace`

* Add handbook IP authentication in hybrid mode

* Fix player level up not displaying on the client properly

* Format code [skip actions]

* Fix game time locking

* Format code [skip actions]

* Update player properties

* Format code [skip actions]

* Move `warn`s for groups to `debug`

* Fix player pausing

* Move more logs to `trace`

* Use `removeItemById` for deleting items via quests

* Clean up logger more

* Pause in-game time when the world is paused

* Format code [skip actions]

* More player property documentation

* Multi-threaded resource loading

* Format code [skip actions]

* Add quest widgets

* Add quests page (basic impl.)

* Add/fix colors

also fix tailwind

* Remove banned packets

client modifications already perform the job of blocking malicious packets from being executed, no point in having this if self-windy is wanted

* Re-add `BeginCameraSceneLookNotify`

* Fix being unable to attack (#2157)

* Add `PlayerOpenChestEvent`

* Add methods to get players from the server

* Add static methods to register an event handler

* Add `PlayerEnterDungeonEvent`

* Remove legacy documentation from `PlayerMoveEvent`

* Add `PlayerChatEvent`

* Add defaults to `Position`

* Clean up `.utils`

* Revert `Multi-threaded resource loading`

* Fix changing target UID when talking to the server

* Lint Code [skip actions]

* Format code [skip actions]

* fix NPC talk triggering main quest in 46101 (#2158)

Make it so that only talks where the param matches the talkId are checked.

* Format code [skip actions]

* Partially fix Chasing Shadows (#2159)

* Partially fix Chasing Shadows

* Go ahead and move it before the return before Magix tells me to.

* Format code [skip actions]

* Bring back period lol (#2160)

* Disable SNI for the HTTPS server

* Add `EntityCreationEvent`

* Add initial startup message

this is so the server appears like its preparing to start

* Format code [skip actions]

* Enable debug mode for plugin loggers if enabled for the primary logger

* Add documentation about `WorldAreaConfigData`

* Make more fields in excels accessible

* Remove deprecated fields from `GetShopRsp`

* Run `spotlessApply` on definitions

* Add `PlayerEnterAreaEvent`

* Optimize event calls

* Fix event invokes

* Format code [skip actions]

* Remove manual autofinish for main quests. (#2162)

* Add world areas to the textmap cache

* Format code [skip actions]

* Don't overdefine variables in extended classes (#2163)

* Add dumper for world areas

* Format code [skip actions]

* instantiate personalLineList (#2165)

* Fix protocol definitions

thank you Nazrin! (+ hiro for raw definitions)

* Fix the background color leaking from the character widget

* Change HTML spacing to 2 spaces

* Implement hiding widgets

* Change scrollbar to a vibrant color

* Add _some_ scaling to the home buttons and its text

* Build the handbook with Gradle

* Fix the 'finer details' with the handbook UI

* Lint Code [skip actions]

* Fix target destination for the Gradle-built handbook

* Implement fetching a player across servers & Add a chainable JsonObject

useful for plugins! might be used in grasscutter eventually

* Fix GitHub actions

* Fix event calling & canceling

* Run `spotlessApply`

* Rename fields (might be wrong)

* Add/update all/more protocol definitions

* Add/update all/more protocol definitions

* Remove outdated packet

* Fix protocol definitions

* Format code [skip actions]

* Implement some lua variables for less console spam (#2172)

* Implement some lua variables for less console spam

* Add GetHostQuestState

This fixes some chapter 3 stuff.

* Format code [skip actions]

* Fix merge import

* Format code [skip actions]

* Fully fix fairy clock for real this time (#2167)

* Fully fix fairy clock For real this time

* Make it so relogging keeps the time lock state.

* Refactor out questLockTime

* Per Hartie, the client packet needs to be changed too

* Update src/main/java/emu/grasscutter/game/world/World.java

Co-authored-by: Magix <27646710+KingRainbow44@users.noreply.github.com>

* Update src/main/java/emu/grasscutter/server/packet/recv/HandlerClientLockGameTimeNotify.java

* Remove all code not needed to get clock working

---------

Co-authored-by: Magix <27646710+KingRainbow44@users.noreply.github.com>

* Implement a proper ability system (#2166)

* Apply fix `21dec2fe`

* Apply fix `89d01d5f`

* Apply fix `d900f154`

this one was already implemented; updated to use call from previous commit

* Ability changing commit

TODO: change info to debug

* Remove use of deprecated methods/fields

* Temp commit v2
(Adding LoseHP and some fixes)

* Oopsie

* Probably fix monster battle

* Fix issue with reflecting into fields

* Fix some things

* Fix ability names for 3.6 resources

* Improve logging

---------

Co-authored-by: StartForKiller <jesussanz2003@gmail.com>

* Format code [skip actions]

* Add system for sending messages between servers

* Format some code

* Remove protocol definitions from Spotless

* Default debug to false; enable with `-debug`

* Implement completely useless global value copying

* HACK: Return the avatar which holds the weapon when the weapon is referred to by ID

* Add properties to `AbilityModifier`

* Change the way HTML is served after authentication

* Use thread executors to speed up the database loading process

* Format code [skip actions]

* Add system for setting handbook address and port

* Lint Code [skip actions]

* Format code [skip actions]

* Fix game-related data not saving

* Format code [skip actions]

* Fix handbook server details

* Lint Code [skip actions]

* Format code [skip actions]

* Use the headers provided by a context to get the IP address

should acknowledge #1975

* Format code [skip actions]

* Move more logs to `trace`

* Format code [skip actions]

* more trace

* Fix something and implement weapon entities

* Format code [skip actions]

* Fix `EntityWeapon`

* Remove deprecated API & Fix resource checking

* Fix unnecessary warning for first-time setup

* Implement handbook request limiting

* Format code [skip actions]

* Fix new avatar weapons being null

* Format code [skip actions]

* Fix issue with 35303 being un-completable & Try to fix fulfilled quest conditions being met

* Load activity config on server startup

* Require plugins to specify an API version and match with the server

* Add default open state ignore list

* Format code [skip actions]

* Quick fix for questing, needs more investigation
This would make the questing work again

* Remove existing hack for 35303

* Fix ignored open states from being set

* Format code [skip actions]

* fix the stupidest bug ive ever seen

* Optimize player kicking on server close

* Format code [skip actions]

* Re-add hack to fix 35303

* Update GitHub actions

* Format code [skip actions]

* Potentially fix issues with regions

* Download additional handbook data

* Revert "Potentially fix issues with regions"

This reverts commit 84e3823695.

---------

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: scooterboo <lewasite@yahoo.com>
Co-authored-by: Tesutarin <105267106+Tesutarin@users.noreply.github.com>
Co-authored-by: Scald <104459145+Arikatsu@users.noreply.github.com>
Co-authored-by: StartForKiller <jesussanz2003@gmail.com>

src/main/java/emu/grasscutter/scripts/serializer/LuaSerializer.java

@@ -4,21 +4,26 @@ import com.esotericsoftware.reflectasm.ConstructorAccess;
 import com.esotericsoftware.reflectasm.MethodAccess;
 import emu.grasscutter.Grasscutter;
 import emu.grasscutter.scripts.ScriptUtils;
+import java.lang.reflect.Field;
+import java.lang.reflect.ParameterizedType;
+import java.lang.reflect.Type;
+import java.util.*;
+import java.util.concurrent.ConcurrentHashMap;
 import lombok.AccessLevel;
 import lombok.AllArgsConstructor;
 import lombok.Data;
 import lombok.experimental.FieldDefaults;
+import org.jetbrains.annotations.Nullable;
 import org.luaj.vm2.LuaTable;
 import org.luaj.vm2.LuaValue;
 
-import java.util.*;
-import java.util.concurrent.ConcurrentHashMap;
-
 public class LuaSerializer implements Serializer {
 
-    private final static Map<Class<?>, MethodAccess> methodAccessCache = new ConcurrentHashMap<>();
-    private final static Map<Class<?>, ConstructorAccess<?>> constructorCache = new ConcurrentHashMap<>();
-    private final static Map<Class<?>, Map<String, FieldMeta>> fieldMetaCache = new ConcurrentHashMap<>();
+    private static final Map<Class<?>, MethodAccess> methodAccessCache = new ConcurrentHashMap<>();
+    private static final Map<Class<?>, ConstructorAccess<?>> constructorCache =
+            new ConcurrentHashMap<>();
+    private static final Map<Class<?>, Map<String, FieldMeta>> fieldMetaCache =
+            new ConcurrentHashMap<>();
 
     @Override
     public <T> List<T> toList(Class<T> type, Object obj) {
@@ -27,7 +32,7 @@ public class LuaSerializer implements Serializer {
 
     @Override
     public <T> T toObject(Class<T> type, Object obj) {
-        return serialize(type, (LuaTable) obj);
+        return serialize(type, null, (LuaTable) obj);
     }
 
     @Override
@@ -35,8 +40,8 @@ public class LuaSerializer implements Serializer {
         return serializeMap(type, (LuaTable) obj);
     }
 
-    private <T> Map<String,T> serializeMap(Class<T> type, LuaTable table) {
-        Map<String,T> map = new HashMap<>();
+    private <T> Map<String, T> serializeMap(Class<T> type, LuaTable table) {
+        Map<String, T> map = new HashMap<>();
 
         if (table == null) {
             return map;
@@ -51,7 +56,7 @@ public class LuaSerializer implements Serializer {
                     T object = null;
 
                     if (keyValue.istable()) {
-                        object = serialize(type, keyValue.checktable());
+                        object = serialize(type, null, keyValue.checktable());
                     } else if (keyValue.isint()) {
                         object = (T) (Integer) keyValue.toint();
                     } else if (keyValue.isnumber()) {
@@ -65,7 +70,7 @@ public class LuaSerializer implements Serializer {
                     }
 
                     if (object != null) {
-                        map.put(String.valueOf(k),object);
+                        map.put(String.valueOf(k), object);
                     }
                 } catch (Exception ex) {
 
@@ -94,7 +99,7 @@ public class LuaSerializer implements Serializer {
                     T object = null;
 
                     if (keyValue.istable()) {
-                        object = serialize(type, keyValue.checktable());
+                        object = serialize(type, null, keyValue.checktable());
                     } else if (keyValue.isint()) {
                         object = (T) (Integer) keyValue.toint();
                     } else if (keyValue.isnumber()) {
@@ -121,12 +126,24 @@ public class LuaSerializer implements Serializer {
         return list;
     }
 
-    public <T> T serialize(Class<T> type, LuaTable table) {
+    private Class<?> getListType(Class<?> type, @Nullable Field field) {
+        if (field == null) {
+            return type.getTypeParameters()[0].getClass();
+        }
+        Type fieldType = field.getGenericType();
+        if (fieldType instanceof ParameterizedType) {
+            return (Class<?>) ((ParameterizedType) fieldType).getActualTypeArguments()[0];
+        }
+
+        return null;
+    }
+
+    public <T> T serialize(Class<T> type, @Nullable Field field, LuaTable table) {
         T object = null;
 
         if (type == List.class) {
             try {
-                Class<T> listType = (Class<T>) type.getTypeParameters()[0].getClass();
+                Class<?> listType = getListType(type, field);
                 return (T) serializeList(listType, table);
             } catch (Exception e) {
                 e.printStackTrace();
@@ -158,7 +175,10 @@ public class LuaSerializer implements Serializer {
                     LuaValue keyValue = table.get(k);
 
                     if (keyValue.istable()) {
-                        methodAccess.invoke(object, fieldMeta.index, serialize(fieldMeta.getType(), keyValue.checktable()));
+                        methodAccess.invoke(
+                                object,
+                                fieldMeta.index,
+                                serialize(fieldMeta.getType(), fieldMeta.getField(), keyValue.checktable()));
                     } else if (fieldMeta.getType().equals(float.class)) {
                         methodAccess.invoke(object, fieldMeta.index, keyValue.tofloat());
                     } else if (fieldMeta.getType().equals(int.class)) {
@@ -171,7 +191,7 @@ public class LuaSerializer implements Serializer {
                         methodAccess.invoke(object, fieldMeta.index, keyValue.tojstring());
                     }
                 } catch (Exception ex) {
-                    //ex.printStackTrace();
+                    // ex.printStackTrace();
                     continue;
                 }
             }
@@ -190,7 +210,8 @@ public class LuaSerializer implements Serializer {
         if (!constructorCache.containsKey(type)) {
             constructorCache.putIfAbsent(type, ConstructorAccess.get(type));
         }
-        var methodAccess = Optional.ofNullable(methodAccessCache.get(type)).orElse(MethodAccess.get(type));
+        var methodAccess =
+                Optional.ofNullable(methodAccessCache.get(type)).orElse(MethodAccess.get(type));
         methodAccessCache.putIfAbsent(type, methodAccess);
 
         var fieldMetaMap = new HashMap<String, FieldMeta>();
@@ -198,20 +219,26 @@ public class LuaSerializer implements Serializer {
 
         Arrays.stream(type.getDeclaredFields())
                 .filter(field -> methodNameSet.contains(getSetterName(field.getName())))
-                .forEach(field -> {
-                    var setter = getSetterName(field.getName());
-                    var index = methodAccess.getIndex(setter);
-                    fieldMetaMap.put(field.getName(), new FieldMeta(field.getName(), setter, index, field.getType()));
-                });
+                .forEach(
+                        field -> {
+                            var setter = getSetterName(field.getName());
+                            var index = methodAccess.getIndex(setter);
+                            fieldMetaMap.put(
+                                    field.getName(),
+                                    new FieldMeta(field.getName(), setter, index, field.getType(), field));
+                        });
 
         Arrays.stream(type.getFields())
                 .filter(field -> !fieldMetaMap.containsKey(field.getName()))
                 .filter(field -> methodNameSet.contains(getSetterName(field.getName())))
-                .forEach(field -> {
-                    var setter = getSetterName(field.getName());
-                    var index = methodAccess.getIndex(setter);
-                    fieldMetaMap.put(field.getName(), new FieldMeta(field.getName(), setter, index, field.getType()));
-                });
+                .forEach(
+                        field -> {
+                            var setter = getSetterName(field.getName());
+                            var index = methodAccess.getIndex(setter);
+                            fieldMetaMap.put(
+                                    field.getName(),
+                                    new FieldMeta(field.getName(), setter, index, field.getType(), field));
+                        });
 
         fieldMetaCache.put(type, fieldMetaMap);
         return fieldMetaMap;
@@ -230,10 +257,11 @@ public class LuaSerializer implements Serializer {
     @Data
     @AllArgsConstructor
     @FieldDefaults(level = AccessLevel.PRIVATE)
-    static class FieldMeta{
+    static class FieldMeta {
         String name;
         String setter;
         int index;
         Class<?> type;
+        @Nullable Field field;
     }
 }

src/main/java/emu/grasscutter/scripts/serializer/LuaTableJacksonSerializer.java

@@ -5,15 +5,15 @@ import com.fasterxml.jackson.annotation.Nulls;
 import com.fasterxml.jackson.core.JsonGenerator;
 import com.fasterxml.jackson.core.JsonProcessingException;
 import com.fasterxml.jackson.databind.*;
+import com.fasterxml.jackson.databind.json.JsonMapper;
 import com.fasterxml.jackson.databind.module.SimpleModule;
 import com.fasterxml.jackson.databind.type.CollectionType;
 import com.fasterxml.jackson.databind.type.MapType;
 import emu.grasscutter.Grasscutter;
-import org.luaj.vm2.LuaTable;
-import org.luaj.vm2.LuaValue;
-
 import java.io.IOException;
 import java.util.*;
+import org.luaj.vm2.LuaTable;
+import org.luaj.vm2.LuaValue;
 
 public class LuaTableJacksonSerializer extends JsonSerializer<LuaTable> implements Serializer {
 
@@ -21,19 +21,24 @@ public class LuaTableJacksonSerializer extends JsonSerializer<LuaTable> implemen
 
     public LuaTableJacksonSerializer() {
         if (objectMapper == null) {
-            objectMapper = new ObjectMapper();
-            objectMapper.configure(MapperFeature.PROPAGATE_TRANSIENT_MARKER, true);
-            // Some properties in Lua table but not in java field
-            objectMapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);
-            objectMapper.configOverride(List.class).setSetterInfo(JsonSetter.Value.forContentNulls(Nulls.AS_EMPTY));
-            SimpleModule luaSerializeModule = new SimpleModule();
+            objectMapper =
+                    JsonMapper.builder()
+                            .configure(MapperFeature.PROPAGATE_TRANSIENT_MARKER, true)
+                            .configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false)
+                            .build();
+            objectMapper
+                    .configOverride(List.class)
+                    .setSetterInfo(JsonSetter.Value.forContentNulls(Nulls.AS_EMPTY));
+
+            var luaSerializeModule = new SimpleModule();
             luaSerializeModule.addSerializer(LuaTable.class, this);
             objectMapper.registerModule(luaSerializeModule);
         }
     }
 
     @Override
-    public void serialize(LuaTable value, JsonGenerator gen, SerializerProvider serializers) throws IOException {
+    public void serialize(LuaTable value, JsonGenerator gen, SerializerProvider serializers)
+            throws IOException {
         if (value == null || value.isnil()) {
             gen.writeNull();
             return;
@@ -117,9 +122,12 @@ public class LuaTableJacksonSerializer extends JsonSerializer<LuaTable> implemen
             return list;
         }
 
-        CollectionType collectionType = objectMapper.getTypeFactory().constructCollectionType(ArrayList.class, type);
+        CollectionType collectionType =
+                objectMapper.getTypeFactory().constructCollectionType(ArrayList.class, type);
         JsonNode jsonNode = objectMapper.valueToTree(luaTable);
-        Grasscutter.getLogger().trace("[LuaTableToList] className={},data={}", type.getCanonicalName(), jsonNode.toString());
+        Grasscutter.getLogger()
+                .trace(
+                        "[LuaTableToList] className={},data={}", type.getCanonicalName(), jsonNode.toString());
         if (jsonNode.isEmpty()) {
             return list;
         }
@@ -151,7 +159,11 @@ public class LuaTableJacksonSerializer extends JsonSerializer<LuaTable> implemen
         }
 
         JsonNode jsonNode = objectMapper.valueToTree(luaTable);
-        Grasscutter.getLogger().trace("[LuaTableToObject] className={},data={}", type.getCanonicalName(), jsonNode.toString());
+        Grasscutter.getLogger()
+                .trace(
+                        "[LuaTableToObject] className={},data={}",
+                        type.getCanonicalName(),
+                        jsonNode.toString());
         try {
             return objectMapper.treeToValue(jsonNode, type);
         } catch (JsonProcessingException e) {
@@ -167,9 +179,12 @@ public class LuaTableJacksonSerializer extends JsonSerializer<LuaTable> implemen
             return map;
         }
 
-        MapType mapStringType = objectMapper.getTypeFactory().constructMapType(HashMap.class, String.class, type);
+        MapType mapStringType =
+                objectMapper.getTypeFactory().constructMapType(HashMap.class, String.class, type);
         JsonNode jsonNode = objectMapper.valueToTree(luaTable);
-        Grasscutter.getLogger().trace("[LuaTableToMap] className={},data={}", type.getCanonicalName(), jsonNode.toString());
+        Grasscutter.getLogger()
+                .trace(
+                        "[LuaTableToMap] className={},data={}", type.getCanonicalName(), jsonNode.toString());
         try {
             Object o = objectMapper.treeToValue(jsonNode, mapStringType);
             if (o != null) {

src/main/java/emu/grasscutter/scripts/serializer/Serializer.java

@@ -3,12 +3,11 @@ package emu.grasscutter.scripts.serializer;
 import java.util.List;
 import java.util.Map;
 
-
 public interface Serializer {
 
-    public <T> List<T> toList(Class<T> type, Object obj);
+    <T> List<T> toList(Class<T> type, Object obj);
 
-    public <T> T toObject(Class<T> type, Object obj);
+    <T> T toObject(Class<T> type, Object obj);
 
-    public <T> Map<String,T> toMap(Class<T> type, Object obj);
+    <T> Map<String, T> toMap(Class<T> type, Object obj);
 }