Merge pull request #2 from padraigfl/level0mod

Alternative modding system to avoid full database refresh

Author: padraigfl

.gitignore

@@ -2,6 +2,8 @@
 *.apk
 *.idsig
 *.split194
+*.split3
+*.split4
 public
 decompiled
 dataconsult

readme.md

@@ -29,13 +29,18 @@ adb is also baked in to simplify the process of connecting to an Android device;
 
 ## Setup:
 
-1. Add a key and certificate to the same directory as you run the jar from for signing the new APK (I may include a dummy one to remove this step)
 1. Run `java -jar [app path]/DropmixModdingTool.jar` (depending on your device you may only need to click on the icon in your Finder/Explorer)
 1. Add a valid Dropmix APK file (must be v1.9.0) and verify it
 1. (if desired) connect Android device (will fail if more than one available)
 1. Go to playlists tab and select the playlists you wish to swap
-1. Build the modded version of the APK
+1. Build the modded version of the APK (Safe Swap recommended; see Swap info below)
 1. Either install it directly to you ADB connected device or save it to your hard drive
+1. Once you have all card data downloaded, you should only run the app with wifi and data disabled to prevent constant server data refreshes
+
+### Recommendations:
+
+- As modifying the apk requires the app to be be signed with a new security key, there will be issues with any data you already have. With this in mind it's recommended to use the "Build Re-Signed APK" function to have a straight copy of the app unmodified but with the same signature as other mods
+- The Safe Swap modification process isn't perfect but as it doesn't break the data integrity within the app (requiring all card data to be redownloaded), it's a better long term option
 
 ## Troubleshooting
 
@@ -45,13 +50,13 @@ I need to know the version of Java you're using and the output of the log panel
 - what version of java is it?
 - do you have multiple adb devices conneccted to your computer?
 - is the Dropmix APK file definitely 1.9.0
+- download all card data via the app and proceed to never use the app while connected to the internet again
 
 ### Known Issues:
 
-1. A fresh install of all card data is required after each install currently [I'm guessing this is some kind of auto refresh function baked into the app]
+1. Full card swap process triggers redownload of all card assets
 1. I raced through this so the UI is janky as hell. I expect frequent restarts. Some of it looks pretty bad with the Java 1.8 compilation settings but it'll do
-1. Logs aren't updating correctly; I think I need to add some multithreading functionality
-1. UI in general has weird lags tbh
+1. UI in general has weird lags tbh; it may need a rethink in general
 1. Need to figure out the licensing stuff fully. My understanding is that all the libraries I depend on here use Apache 2.0 so I will use that too
 1. The included keys for signing the APK currently have had less than zero thought put into them. This is a rough and insecure project. If you want to improve anything around the security of this project I'd love the help!
 1. No data is saved between instances of the program so you need to readd the apk each time
@@ -60,14 +65,30 @@ I need to know the version of Java you're using and the output of the log panel
 ## Current functionality:
 
 1. Parse app's key card database
-1. Generates a modified version of the application with playlists swapped
+1. Generates a modified version of the application with playlists swapped (two processes available)
 1. Directly installs modified APK onto connected android device (requires ADB server instance)
 
+### Modification Options
+
+#### Full swap
+
+This simply swaps the card IDs on the top level data tables (found in sharedassets), meaning a card will behave exactly like its swapped counterpart.
+
+Unfortunately this means the card's have data which does not match the app data and a fresh download of assets will be triggered before the game can be played; meaning limited long term value
+
+#### Safe Swap
+
+This swaps the IDs in the game level data (found in level0), mostly behaving exactly the same however "Power" appears to come from the top level database.
+
+#### Future Options
+
+As I've had to update the game level data for the safe swap, I've largely implemented the parsing and writing of the database which would allow custom cards along with more extensive game modifications (e.g. FX cards rules appear to be set here too)
+
 ## Future work:
 
-1. Alternative swap process using `level0` data (will not require fresh data downloads so future-proof but won't have 100% accurate card behaviour)
-1. Include associated baffler cards in playlist swaps
-1. Mod iOS app on M1 devices
-1. Verify working on Windows
-1. Tools for straightforward installing of APK without mods
-1. Mayyyyybe custom cards? This involves modifying the level0 assets file instead and injecting custom data files post-install so it's not so easy
+This is the work I intend to do rather than stretch goals and bolder things
+
+1. Output APK alongside CSV file outlining changes from the main one
+1. Verify working on Windows (partially checked, ADB not)
+1. Tools for straightforward installing of APK without mods (i.e. install APK to device, copy data files over)
+1. Mod iOS app on M1 devices 

src/AdbWinApi.dll

@@ -0,0 +1,49 @@
+package model;
+
+import util.Helpers;
+
+/*
+Can be used for:
+- shared assets card data bases (each season is one record)
+- level0 in-game card data (each row in table is one record)
+- probably other locations
+ */
+public abstract class AbstractDropmixDataRecord {
+  int startIdx;
+  int dataStartIdx;
+  int dataLength;
+  int dataSpace; // the space occupied by the data (does not include the specifier before the data, does include whitespace at end)
+  byte[] raw;// everything including the 32 bit length specifier and whitespace
+  byte[] recordData; //
+  boolean iOS;
+  boolean modified;
+  public AbstractDropmixDataRecord(byte[] data, int startIdx, boolean iOS) {
+    this.startIdx = startIdx;
+    this.dataStartIdx = startIdx + 4;
+    this.dataLength = Helpers.intFromByteArray(Helpers.getNRange(data, startIdx, 4));
+    int remainder = (dataLength) % 4;
+    this.dataSpace = dataLength + (remainder == 0 ? 0 : 4 - remainder);
+    this.raw = Helpers.getNRange(data, startIdx, dataSpace + 4);
+    this.recordData = Helpers.getNRange(data, startIdx + 4, dataLength);
+    this.iOS = iOS;
+  }
+
+  public String toString() {
+    return (this.modified ? "[MODIFIED]" : "") + Helpers.byteArrayToString(Helpers.getNRange(raw, 4, dataLength));
+  }
+
+  public static int getStartIndex(byte[] rawData, byte[] startSequence) {
+    outer:
+    for (int i = 0; i < rawData.length; i++) {
+      if (rawData[i] == startSequence[0]) {
+        for (int j = 1; j < startSequence.length &&  (j+i) < rawData.length; j++ ) {
+          if (rawData[i + j] != startSequence[j]) {
+            continue outer;
+          }
+        }
+        return i + startSequence.length;
+      }
+    }
+    return -1;
+  }
+}

src/AdbWinUsbApi.dll

@@ -2,28 +2,25 @@
 
 import se.vidstige.jadb.JadbDevice;
 import ui.UIMain;
+import util.Helpers;
 
-import javax.swing.*;
 import java.io.File;
-import java.io.IOException;
-import java.nio.file.Files;
-import java.nio.file.Path;
-import java.nio.file.Paths;
 import java.util.*;
 
 public class AppState {
   public static AppState instance = new AppState();
   public File apkFile;
   public File dataZip;
-  public byte[] rawData;
-  public TreeMap<String, String> swapOptions = new TreeMap<String, String>();;
+  public TreeMap<String, String> swapOptions = new TreeMap<>();
   public TreeMap<String, String> playlistSwap = new TreeMap<>();
   public JadbDevice adbDevice;
   public DropmixSharedAssets assetsHandler;
+  public DropmixLevel0 level0Handler;
   public LogOptions logState = LogOptions.ERROR;
   public Process currentProcess = Process.NONE;
   public UIMain appFrame; // for forcing refreshes
   public boolean isNestedLog;
+  public boolean iOS = false;
   private AppState() {
   }
   public static AppState getInstance() {
@@ -32,7 +29,7 @@ public static AppState getInstance() {
   public static AppState getInstance(boolean isTest) {
     AppState instance = getInstance();
     if (isTest) {
-      instance.setData(instance.loadFile());
+      instance.setData(Helpers.loadFile("sharedassets0.assets.split194"), Helpers.loadFile("level0.split3"));
     }
     return instance;
   }
@@ -41,43 +38,43 @@ public static AppState getInstance(boolean isTest, UIMain appFrame) {
     instance.appFrame = appFrame;
     return instance;
   }
-  public void setData(byte[] fileData) {
-    this.assetsHandler = new DropmixSharedAssets(fileData);
+  public void setData(byte[] sharedAssets, byte[] level0) {
+    this.assetsHandler = new DropmixSharedAssets(sharedAssets);
+    this.level0Handler = new DropmixLevel0(level0);
   }
 
-  public CardDetail[] getCards() {
-    ArrayList<CardDetail> cards = new ArrayList<CardDetail>();
+  public DropmixSharedAssetsCard[] getCards() {
+    ArrayList<DropmixSharedAssetsCard> cards = new ArrayList<>();
     try {
       int seasonIdx = 0;
-      SeasonTable season = this.assetsHandler.seasons.get(seasonIdx++);
+      DropmixSharedAssetsSeason season = this.assetsHandler.seasons.get(seasonIdx++);
       while (season != null) {
         cards.addAll(Arrays.asList(season.cards));
         season = this.assetsHandler.seasons.get(seasonIdx++);
       }
-      return cards.toArray(new CardDetail[0]);
+      return cards.toArray(new DropmixSharedAssetsCard[0]);
     } catch (Exception e) {
       e.printStackTrace();
-      return new CardDetail[0];
+      return new DropmixSharedAssetsCard[0];
     }
   }
-  public PlaylistDetail[] getPlaylists() {
-    Set<String> playlistNames = new HashSet<String>();
-    for (CardDetail c: AppState.getInstance().getCards()) {
-      playlistNames.add(c.cardData.get(CardDetail.SeriesIcon));
+  public DropmixSharedAssetsPlaylist[] getPlaylists() {
+    Set<String> playlistNames = new HashSet<>();
+    for (DropmixSharedAssetsCard c: AppState.getInstance().getCards()) {
+      playlistNames.add(c.data.get(DropmixSharedAssetsCard.SeriesIcon));
     }
     String[] playlistNamesArray = playlistNames.toArray(new String[0]);
 
 
-    ArrayList<PlaylistDetail> seasons = new ArrayList<>();
+    ArrayList<DropmixSharedAssetsPlaylist> seasons = new ArrayList<>();
 
     for (int i=0; i < playlistNames.size(); i++) {
-      PlaylistDetail playlist = new PlaylistDetail(playlistNamesArray[i]);
+      DropmixSharedAssetsPlaylist playlist = new DropmixSharedAssetsPlaylist(playlistNamesArray[i]);
       seasons.add(playlist);
     }
     // this is required to sort the playlists in the common order
-    Collections.sort(seasons, new Comparator<PlaylistDetail>(){
-      public int compare(PlaylistDetail o1, PlaylistDetail o2)
-      {
+    seasons.sort(new Comparator<DropmixSharedAssetsPlaylist>() {
+      public int compare(DropmixSharedAssetsPlaylist o1, DropmixSharedAssetsPlaylist o2) {
         int val = o1.season.compareTo(o2.season);
         if (val == 0) {
           // baffler and promo are both empty
@@ -89,26 +86,15 @@ public int compare(PlaylistDetail o1, PlaylistDetail o2)
           }
           int card1 = Integer.parseInt(o1.cardId);
           int card2 = Integer.parseInt(o2.cardId);
+          if (card1 == card2) {
+            return 0;
+          }
           return card1 > card2 ? 1 : -1;
         }
         return val;
       }
     });
-    return seasons.toArray(new PlaylistDetail[0]);
-  }
-  byte[] loadFile() {
-    if (rawData != null) {
-      return rawData;
-    }
-    ClassLoader classLoader = getClass().getClassLoader();
-
-    try {
-      String fileByteArrayPathString = classLoader.getResource("sharedassets0.assets.split194").getFile();
-      rawData = Files.readAllBytes(Paths.get(fileByteArrayPathString));
-      return rawData;
-    } catch (IOException | NullPointerException e) {
-      throw new Error(e);
-    }
+    return seasons.toArray(new DropmixSharedAssetsPlaylist[0]);
   }
   public void removePlaylistSwap(String p1) {
     String p2 = this.playlistSwap.get(p1);
@@ -128,8 +114,8 @@ public void setPlaylistSwap(String p1, String p2) throws Exception {
     if (this.playlistSwap.containsValue(p1) || this.playlistSwap.containsValue(p2)) {
       throw new Exception("value-in-use");
     }
-    for(PlaylistDetail pl : this.getPlaylists()) {
-      if (pl.name == p1 || pl.name == p2) {
+    for(DropmixSharedAssetsPlaylist pl : this.getPlaylists()) {
+      if (pl.name.equals(p1) || pl.name.equals(p2)) {
         if (pl.playlistCount != 15) {
           throw new Exception("invalid-playlist");
         }
@@ -195,25 +181,25 @@ public void reset() {
     this.appFrame.addPlaceholders();
   }
   // builds a card based swap from the playlist swap; may require more careful refinement as assumptions about order persistence exist
-  public static TreeMap<String, String> getCardSwapFromPlaylist(TreeMap<String, String> plSwap) {
+  public static TreeMap<String, String> getCardSwapFromPlaylist(TreeMap<String, String> plSwap, boolean includeBafflers) {
     for (String key: plSwap.values()) {
       String value = plSwap.get(key);
       String validator = plSwap.get(value);
-      if (value == null || validator == null || !key.equals(validator)) {
+      if (value == null || !key.equals(validator)) {
         throw new RuntimeException("playlist-swap-sync-issue");
       }
     }
     String[] playlistNames = plSwap.keySet().toArray(new String[0]);
-    PlaylistDetail[] playlists = getInstance().getPlaylists();
-    TreeMap<String, PlaylistDetail> playlistDetailTreeMap = new TreeMap<String, PlaylistDetail>();
+    DropmixSharedAssetsPlaylist[] playlists = getInstance().getPlaylists();
+    TreeMap<String, DropmixSharedAssetsPlaylist> dropmixSharedAssetsPlaylistTreeMap = new TreeMap<>();
     TreeMap<String, String> generatedCardSwap = new TreeMap<>();
     Set<String> alreadySwappedPlaylists = new HashSet<>();
-    for (PlaylistDetail pl: playlists) {
-      playlistDetailTreeMap.put(pl.name, pl);
+    for (DropmixSharedAssetsPlaylist pl: playlists) {
+      dropmixSharedAssetsPlaylistTreeMap.put(pl.name, pl);
     }
     for (String playlist: playlistNames) {
-      PlaylistDetail srcPl = playlistDetailTreeMap.get(playlist);
-      PlaylistDetail swapPl = playlistDetailTreeMap.get(plSwap.get(playlist));
+      DropmixSharedAssetsPlaylist srcPl = dropmixSharedAssetsPlaylistTreeMap.get(playlist);
+      DropmixSharedAssetsPlaylist swapPl = dropmixSharedAssetsPlaylistTreeMap.get(plSwap.get(playlist));
       if (alreadySwappedPlaylists.contains(swapPl.name)) {
         continue;
       }
@@ -225,7 +211,15 @@ public static TreeMap<String, String> getCardSwapFromPlaylist(TreeMap<String, St
           generatedCardSwap.put(srcPl.cards[i], swapPl.cards[i]);
           generatedCardSwap.put(swapPl.cards[i], srcPl.cards[i]);
         } catch (Exception e) {
-          continue;
+          e.printStackTrace();
+        }
+      }
+      if (includeBafflers) {
+        String srcBaffler = srcPl.getBaffler();
+        String swapBaffler = swapPl.getBaffler();
+        if (srcBaffler != null && swapBaffler != null) {
+          generatedCardSwap.put(srcBaffler, swapBaffler);
+          generatedCardSwap.put(swapBaffler, srcBaffler);
         }
       }
       alreadySwappedPlaylists.add(playlist);

src/level0.split3

@@ -14,7 +14,7 @@ public void getCardSwapFromPlaylist() {
     plSwap.put("highness", "sweets");
     plSwap.put("sweets", "highness");
 
-    TreeMap<String, String> cardSwap = AppState.getCardSwapFromPlaylist(plSwap);
+    TreeMap<String, String> cardSwap = AppState.getCardSwapFromPlaylist(plSwap, false);
     Assertions.assertEquals(cardSwap.size(), 30);
     Assertions.assertTrue(cardSwap.containsKey("LIC_0058_Wild"));
     Assertions.assertTrue(cardSwap.containsValue("LIC_0058_Wild"));
@@ -25,7 +25,7 @@ public void getCardSwapFromPlaylist() {
       plSwap.clear();
       plSwap.put("baffler", "promo");
       plSwap.put("promo", "baffler");
-      cardSwap = AppState.getCardSwapFromPlaylist(plSwap);
+      AppState.getCardSwapFromPlaylist(plSwap, false);
     } catch (RuntimeException e) {
       errorMessage = e.getMessage();
     }
@@ -37,7 +37,7 @@ public void getCardSwapFromPlaylist() {
       plSwap.put("instinct", "bomb");
       plSwap.put("chiller", "instinct");
       plSwap.put("bomb", "chiller");
-      cardSwap = AppState.getCardSwapFromPlaylist(plSwap);
+      AppState.getCardSwapFromPlaylist(plSwap, false);
     } catch (RuntimeException e) {
       errorMessage = e.getMessage();
     }