--------------------------------------------- | Decomposition of Android Malware Datasets | --------------------------------------------- RedNaga et al. suggested a simple method to quickly fingerprint the compiler used to generate an Android APK archive, suggesting whether an app is repackaged or developed from scratch (i.e., using an IDE and compiled using the SDK compiler). ----------------------------------------------------------- | Dataset | dx | dexmerge | dexlib 1.X + 2.X | Total | ----------------------------------------------------------- | MalGenome | 52% | -- | 48% | 1234 | ----------------------------------------------------------- | Drebin* | 84% | -- | 16% | 4326 | ----------------------------------------------------------- | Play Store | 61% | 34% | 5% | 1882 | ----------------------------------------------------------- | Piggy (mal) | 22% | 6% | 71% | 1399 | ----------------------------------------------------------- | Piggy (good)| 61% | 22% | 17% | 1355 | ----------------------------------------------------------- | AMD | 38% | 35% | 26% | 24526 | ----------------------------------------------------------- ------------------- | Malgenome Stats | ------------------- [*] Total: 1234 [*] Detected by (average): 31.43 [*] Marketplaces: 95% + "genome" (Mostly gathered manually) [*] Multiple types: yes 100% [*] Multiple names: yes 100% [*] Top 10 Names: [('droidkungfu', 471.0), ('basebridge', 308.0), ('geinimi', 67.0), ('kmin', 51.0),('ddlight', 47.0), ('golddream', 46.0), ('pjapps', 45.0), ('lotoor', 23.0), ('yzhc', 22.0), ('adrd',22.0)] [*] Top 10 Types: [('trojan', 1168.0), ('exploit', 42.0), ('spyware', 21.0), ('spy++trojan', 1.0), ('fakeneflic++trojan', 1.0)] ---------------------- | Piggybacking Stats | ---------------------- [*] Total 1136 [*] Detected by (average): 9 [*] Marketplaces: [('anzhi', 727.0), ('appchina', 139.0), ('angeeks', 58.0), ('1mobile', 57.0), ('play.google.com', 50.0), ('genome', 36.0), ('play.google.com|appchina', 8.0), ('anzhi|appchina', 5.0), ('slideme', 4.0), ('mi.com|anzhi', 4.0)] [*] Multiple types: yes (55%) + no (45%) [*] Multiple names: yes (65%) + no (35%) [*] Top 10 names: [('dowgin', 280.0), ('kuguo', 248.0), ('gingermaster', 68.0), ('adwhirlads', 64.0), ('ginermaster', 63.0), ('admobads', 45.0), ('adwo', 34.0), ('youmi', 25.0), ('droidkungfu', 24.0), ('geinimi', 20.0)] [*] Top 10 types: [('adware', 728.0), ('trojan', 284.0), ('spyware', 26.0), ('riskware', 25.0), ('adsware', 25.0), ('troj', 22.0), ('unclassifiedmalware', 6.0), ('adware++adware', 4.0), ('trojansms', 2.0), ('spr', 2.0)] ------- | AMD | ------- [*] Total: 204 (out of 1250) [*] Detected by (average): 24.6 [*] Marketplaces: [('genome', 60.0), ('play.google.com', 56.0), ('appchina', 47.0), ('anzhi', 22.0), ('angeeks', 10.0), ('appchina|play.google.com', 3.0), ('play.google.com|appchina', 1.0), ('freewarelovers', 1.0), ('appchina|genome', 1.0) [*] Multiple types: yes (93%) + no (7%) [*] Multiple names: yes (94%) + no (6%) [*] Top 10 names: [('droidkungfu', 42.0), ('airpush', 18.0), ('ginmaster', 17.0), ('kyview', 14.0), ('dowgin', 13.0), ('golddream', 12.0), ('nandrobox', 11.0), ('lotoor', 11.0), ('youmi', 8.0), ('kuguo', 8.0)] [*] Top 10 types: [('trojan', 87.0), ('adware', 70.0), ('exploit', 23.0), ('riskware', 8.0), ('monitor', 7.0), ('spyware', 4.0), ('hacktool', 2.0), ('fakeupdates++trojan', 1.0), ('addisplay', 1.0)] ---------------------- | Static Experiments | ---------------------- [*] Malgnome+GPlay16: {'recall': 0.971473851030111, 'specificity': 0.9934102141680395, 'f1score': 0.9823717948717949, 'precision': 0.993517017828201, 'accuracy': 0.9822294022617124} [*] Piggy+Original: {'recall': 0.700079554494829, 'specificity': 0.6532258064516129, 'f1score': 0.6633999246136449, 'precision': 0.6303724928366762, 'accuracy': 0.6746812386156649} [*] AMD+GPlay16: {'recall': 0.9263754045307443, 'specificity': 0.9599666388657214, 'f1score': 0.9427748044462742, 'precision': 0.9597652975691534, 'accuracy': 0.9429158110882957} [*] Piggy+Gplay16: {'recall': 0.9273255813953488, 'specificity': 0.9049128367670365, 'f1score': 0.9206349206349206, 'precision': 0.9140401146131805, 'accuracy': 0.9166034874905231} [*] AMD+Original: {'recall': 0.905742145178765, 'specificity': 0.7794935145151328, 'f1score': 0.7901701323251417, 'precision': 0.7007544006705784, 'accuracy': 0.8253343823760818} Train with A, test with B ------------------------- "Forward compatibility" [*] (Train) Malgenome+Gplay16 (Test) Piggy+Original: {'recall': 0.4951012410189419, 'specificity': 0.4744645799011532, 'f1score': 0.5179364537068671, 'precision': 0.5429799426934098, 'accuracy': 0.48597449908925316} [*] (Train) Malgenome+Gplay16 (Test) AMD+Gplay16: {'recall': 0.9596153846153846, 'specificity': 0.8602150537634409, 'f1score': 0.8938647559337215, 'precision': 0.8365465213746857, 'accuracy': 0.9026694045174538} "Lateral compatibility" [*] (Train) AMD+Gplay16 (Test) Piggy+Original: {'recall': 0.5050119331742243, 'specificity': 0.48, 'f1score': 0.6061300486966485, 'precision': 0.7578796561604585, 'accuracy': 0.4990892531876138} [*] (Train) Piggy+Original (Test) AMD+Gplay16: {'recall': 0.4680161943319838, 'specificity': 0.4875, 'f1score': 0.4761120263591434, 'precision': 0.4844928751047779, 'accuracy': 0.47761806981519506} "Backward compability" [*] (Train) AMD+Gplay16 (Test) Malgenome+Gplay16: {'recall': 0.9505791505791505, 'specificity': 0.9974597798475868, 'f1score': 0.9735073151443259, 'precision': 0.9975688816855753, 'accuracy': 0.972940226171244} [*] (Train) Piggy+Original (Test) Malgenome+Gplay16: {'recall': 0.5122850122850123, 'specificity': 0.5084235860409145, 'f1score': 0.4072265625, 'precision': 0.3379254457050243, 'accuracy': 0.5096930533117933} Poisoning Learners ------------------- [*] Piggy+GPlay16+Original: {'recall': 0.7509481668773704, 'specificity': 0.7490613266583229, 'f1score': 0.5432098765432098, 'precision': 0.42550143266475643, 'accuracy': 0.7494356659142212} [*] Piggy+AMD+Original: {'recall': 0.7588488256698643, 'specificity': 0.6775956284153005, 'f1score': 0.8175338560228083, 'precision': 0.8860563924295095, 'accuracy': 0.7399695276790249} [*] AMD+Gplay16+Original: {'recall': 0.9307411907654921, 'specificity': 0.8557919621749409, 'f1score': 0.7599206349206349, 'precision': 0.6420787929589271, 'accuracy': 0.872093023255814} Training Dataset,Test Dataset,Score,Notes amd+gplay,piggybacked,0.81/0.72,Orthodox amd+gplay,original,0.20/0.38,??? amd+original,piggybacked,0.17/0.50,Adversarial amd+original,original,0.98/0.94,Makes sense amd+original,malgenome,0.78/0.83,Makes sense amd+malgenome+gplay,piggybacked,0.81/0.79,Orthodox amd+malgenome+gplay,original,0.20/0.30,??? amd+original+gplay,piggybacked,0.19/0.34,Adversarial amd+original+gplay,original,0.98/0.98,Makes sense amd+original+gplay,malgenome,0.86/0.65,Makes sense amd+malgenome+original+gplay,piggybacked,0.30/0.43,Adversarial amd+malgenome+original+gplay,original,0.91/0.92,Makes sense amd+malgenome+original+gplay,malgenome,0.99/0.95,Makes sense ------------------------------- | Dynamic Experiments (1 run) | ------------------------------- [*] Malgnome+GPlay16: {'recall': 0.9476309226932669, 'specificity': 0.9432924248836225, 'f1score': 0.830601092896175, 'precision': 0.7392996108949417, 'accuracy': 0.9439218523878437} [*] Piggy+Original: {'recall': 0.7065803667745415, 'specificity': 0.6086956521739131, 'f1score': 0.7322526551145891, 'precision': 0.759860788863109, 'accuracy': 0.6710164835164835} [*] AMD+GPlay16: {'recall': 0.8761061946902655, 'specificity': 0.8736681887366818, 'f1score': 0.8661417322834645, 'precision': 0.856401384083045, 'accuracy': 0.8747954173486089} [*] Piggy+Gplay16: {'recall': 0.8550106609808102, 'specificity': 0.8943661971830986, 'f1score': 0.8911111111111112, 'precision': 0.9303944315545244, 'accuracy': 0.8698539176626826} [*] AMD+Original: {'recall': 0.8139963167587477, 'specificity': 0.7837837837837838, 'f1score': 0.7885816235504014, 'precision': 0.7647058823529411, 'accuracy': 0.7977815699658704} Train with A, test with B ------------------------- "Forward compatibility" [*] (Train) Malgenome+Gplay16 (Test) Piggy+Original: {'recall': 0.6524064171122995, 'specificity': 0.44581005586592176, 'f1score': 0.5144061841180605, 'precision': 0.4245939675174014, 'accuracy': 0.5254120879120879} [*] (Train) Malgenome+Gplay16 (Test) AMD+Gplay16: {'recall': 0.9283819628647215, 'specificity': 0.7301775147928994, 'f1score': 0.7329842931937173, 'precision': 0.6055363321799307, 'accuracy': 0.7913256955810147} "Lateral compatibility" [*] (Train) AMD+Gplay16 (Test) Piggy+Original: {'recall': 0.6342685370741483, 'specificity': 0.5, 'f1score': 0.6806451612903226, 'precision': 0.734338747099768, 'accuracy': 0.592032967032967} [*] (Train) Piggy+Original (Test) AMD+Gplay16: {'recall': 0.5771889400921659, 'specificity': 0.7824858757062146, 'f1score': 0.6929460580912863, 'precision': 0.8667820069204152, 'accuracy': 0.6366612111292962} "Backward compability" [*] (Train) AMD+Gplay16 (Test) Malgenome+Gplay16: {'recall': 0.9190751445086706, 'specificity': 0.9420970266040689, 'f1score': 0.9235237173281705, 'precision': 0.9280155642023347, 'accuracy': 0.9317789291882557} [*] (Train) Piggy+Original (Test) Malgenome+Gplay16: {'recall': 0.555045871559633, 'specificity': 0.8951048951048951, 'f1score': 0.6984126984126985, 'precision': 0.9416342412451362, 'accuracy': 0.6390328151986183} Poisoning Learners ------------------- [*] Piggy+GPlay16+Original: {'recall': 0.7962962962962963, 'specificity': 0.7230769230769231, 'f1score': 0.6134094151212552, 'precision': 0.4988399071925754, 'accuracy': 0.741904761904762} [*] Piggy+AMD+Original: {'recall': 0.7519582245430809, 'specificity': 0.6275659824046921, 'f1score': 0.8196371398078975, 'precision': 0.9007036747458952, 'accuracy': 0.7293112653497064} ---------------------------- | Most difficult to detect | ---------------------------- [*] Piggybacking+Original (static): > By name: [('admobads', 41.0), ('dowgin', 35.0), ('kuguo', 21.0), ('wooboo', 16.0), ('youmi', 14.0), ('umeng', 13.0), ('geinimi', 11.0), ('domob', 9.0), ('tapjoyads', 8.0), ('droidkungfu', 8.0)] > By type: [('adware', 199.0), ('trojan', 67.0), ('adsware', 23.0), ('troj', 18.0), ('unclassifiedmalware', 5.0), ('riskware', 4.0), ('spyware', 3.0), ('adware++adware', 3.0), ('exploit', 2.0), ('virus', 1.0)] > By detected: 7 [*] AMD+Original (static): N/A > By name: N/A > By type: N/A > By detected: N/A [*] Piggybacking+Original (dynamic): > By name: [('admobads', 13.0), ('kuguo', 9.0), ('boqx', 6.0), ('adwo', 4.0), ('adwhirlads', 4.0), ('youmi', 3.0), ('umeng', 3.0), ('gingermaster', 3.0), ('droidkungfu', 3.0), ('domob', 3.0)] > By type: [('adware', 41.0), ('trojan', 19.0), ('troj', 8.0), ('adsware', 6.0), ('spyware', 4.0), ('riskware', 3.0), ('adware++adware', 2.0), ('unclassifiedmalware', 1.0), ('rootor++spyware', 1.0)] > By detected: 7 [*] AMD+Original (dynamic): N/A > By name: N/A > By type: N/A > By detected: N/A