[移动开发]Android日志能力之持久化转储及回传的闭环方案（上）

Android日志能力之持久化转储及回传的闭环方案（上）

背景

在现如今Android市场的半壁江山下各ODM厂商OEM厂商谁要是没点获取问题日志的能力那可能会被这快速的更新迭代大浪淘沙掉，尤其是一款新上市的产品初期，必定是有各种各样的售后问题反馈，俗话说-用户即上帝，在上帝反馈前和反馈后及时获取对于开发者有用的全面的信息必定是推动产品软件快速更新迭代和赢得上帝信任好评的关键。

今天我们就来聊聊一种持久化日志转储及回传的闭环设计方案。

Android系统中常用的日志信

众所周知，logcat作为Android中不可或缺的开发者调试利器，让我们能直观的看到我们的程序究竟发生了什么，在哪里跑飞，又是如何崩溃的，甚至于哪个函数的哪一行系统都已经提示出来了。但是对于开发者来说，不可能追到用户家里去拿着电脑调试用户的手机/平板，那就必须要获取出现问题的日志信息，然而就算在出现Crash/ANR后去获取logcat，必然已经不是案发现场了（就算是案发现场也可能不是第一现场了），因此将案发现场进行保存必然是有利于我们分析破案的。

上帝反馈前

在Android系统的设计时Google的开发者必然也有同样的苦恼 —— 如何准确的获取用户程序出现问题的信息。因此机智的他们在framework和native层埋下了一个追踪器 —— DropBox。只要是在framework层处理的异常信息（ANR、CRASH）均会将关键的信息收集到DropBox中，并记录在/data/system/dropbox目录下。

DropBox涵盖了绝大多数的异常和非法操作的关键信息，以系统处理Crash的流程为例，在ActivityManagerService中处理application crash时通过如下流程将关键的信息生成一条增加到DropBoxManager中：

ActivityManagerService.java

    /**
     * Used by {@link com.android.internal.os.RuntimeInit} to report when an application crashes.
     * The application process will exit immediately after this call returns.
     * @param app object of the crashing app, null for the system server
     * @param crashInfo describing the exception
     */
    public void handleApplicationCrash(IBinder app,
            ApplicationErrorReport.ParcelableCrashInfo crashInfo) {
        ProcessRecord r = findAppProcess(app, "Crash");
        final String processName = app == null ? "system_server"
                : (r == null ? "unknown" : r.processName);

        handleApplicationCrashInner("crash", r, processName, crashInfo);
    }
    
    void handleApplicationCrashInner(String eventType, ProcessRecord r, String processName,
        ApplicationErrorReport.CrashInfo crashInfo) {
        ......
        mAmsExt.onNotifyAppCrash(Binder.getCallingPid(), Binder.getCallingUid(),
                        (r != null && r.info != null) ? r.info.packageName : "");
        addErrorToDropBox(
                eventType, r, processName, null, null, null, null, null, null, crashInfo);
        ......
    }

    /**
     * Write a description of an error (crash, WTF, ANR) to the drop box.
     */
    public void addErrorToDropBox(String eventType,
            ProcessRecord process, String processName, String activityShortComponentName,
            String parentShortComponentName, ProcessRecord parentProcess,
            String subject, final String report, final File dataFile,
            final ApplicationErrorReport.CrashInfo crashInfo) {
        ......
    }

调用ActivityManagerService的addErrorToDropBox方法后会生成一条对应的Entry，DropBoxManagerService会对此Entry进行处理，将其记录在/data/system/dropbox目录下，并发送一条广播 —— DropBoxManager.ACTION_DROPBOX_ENTRY_ADDED,并将时间戳与DropBoxTag作为Extra内容：

DropBoxManagerService.java

    private class DropBoxManagerBroadcastHandler extends Handler {
        ......
        private Intent createIntent(String tag, long time) {
            final Intent dropboxIntent = new Intent(DropBoxManager.ACTION_DROPBOX_ENTRY_ADDED);
            dropboxIntent.putExtra(DropBoxManager.EXTRA_TAG, tag);
            dropboxIntent.putExtra(DropBoxManager.EXTRA_TIME, time);
            return dropboxIntent;
        }
        
        /**
         * Schedule a dropbox broadcast to be sent asynchronously.
         */
        public void sendBroadcast(String tag, long time) {
            sendMessage(obtainMessage(MSG_SEND_BROADCAST, createIntent(tag, time)));
        }
        ......
    }

既然巨人已经为我们铺好了路，那我们就站在巨人的肩膀上，直接监听系统发送的DropBox广播即可。

基于DropBox的系统异常信息收集上报方案

因此，在此提出一种异常上报方案，由于DropBoxManagerService实现了崩溃信息转储，并且，DropBox在每转储一次（执行一次add操作）时会发出一个protected broadcast：DropBoxManager.ACTION_DROPBOX_ENTRY_ADDED，应用可以通过此接收此广播后进行崩溃信息解析及上报工作：

ExceptionReporterService.kt

/**
 * @ClassName: ExceptionReporterService
 * @Desc: Main foreground service of exception report.
 * */
class ExceptionReporterService : Service() {
    private lateinit var mDropBoxReceiver: DropBoxReceiver

    override fun onCreate() {
        super.onCreate()
        val intentFilter = IntentFilter()
        intentFilter.addAction(DropBoxManager.ACTION_DROPBOX_ENTRY_ADDED)
        mDropBoxReceiver = DropBoxReceiver()
        registerReceiver(mDropBoxReceiver, intentFilter)
        registerExceptionBootMsgUploader(applicationContext)
    }

    private fun registerExceptionBootMsgUploader(context: Context) {
        val cm = context.getSystemService(Context.CONNECTIVITY_SERVICE) as ConnectivityManager
        val networkBuilder = NetworkRequest.Builder()

        networkBuilder.addTransportType(NetworkCapabilities.TRANSPORT_WIFI)
        networkBuilder.addTransportType(NetworkCapabilities.TRANSPORT_ETHERNET)
        cm.registerNetworkCallback(
            networkBuilder.build(),
            NetworkConnectionCallback(context)
        )
    }
    
    ......

    override fun onDestroy() {
        unregisterReceiver(mDropBoxReceiver)
    }

    override fun onBind(intent: Intent?): IBinder? {
    }
    ......
}

DropBox的TAG较多，分为实时性较低的TAG和实时性较高的TAG，实时性较低的TAG如TAG_SYSTEM_LAST_KMSG、TAG_SYSTEM_BOOT这类通常只有重启开机时才会记录，但实时性较高的TAG如：TAG_SYSTEM_APP_CRASH、TAG_DATA_APP_CRASH等，一旦运行到crash或ANR后则会进行记录，因此利用该特点在方案设计时也将实时性较高的TAG与实时性较低的TAG分离，并且只接受严重的崩溃/无响应异常：

实时性较高的异常上报

DropBoxReceiver.kt

/**
 * @ClassName: DropBoxReceiver
 * @Desc: Start DropBoxEntryAnalysisService when receive broadcast of
 *       'DropBoxManager.ACTION_DROPBOX_ENTRY_ADDED'.
 * */
class DropBoxReceiver : BroadcastReceiver() {

    companion object {
        val uploadTagList = mutableListOf(
            TAG_SYSTEM_APP_CRASH,
            TAG_SYSTEM_APP_NATIVE_CRASH,
            TAG_SYSTEM_APP_ANR,
            TAG_DATA_APP_CRASH,
            TAG_DATA_APP_NATIVE_CRASH,
            TAG_DATA_APP_ANR,
            TAG_SYSTEM_SERVER_NATIVE_CRASH,
            TAG_SYSTEM_SERVER_WATCHDOG,
            TAG_SYSTEM_SERVER_LOWMEM,
            TAG_SYSTEM_TOMBSTONE
        )
        private const val TAG: String = "DropBoxReceiver"
    }

    override fun onReceive(context: Context, intent: Intent) {
        Log.d(TAG, "onReceive: DropBoxManager.ACTION_DROPBOX_ENTRY_ADDED")
        val dropBoxEntryFileTag = intent.getStringExtra(DropBoxManager.EXTRA_TAG)
        val dropBoxEntryFileTime = intent.getLongExtra(DropBoxManager.EXTRA_TIME, 0)

        if (!dropBoxEntryFileTag.isNullOrEmpty() && checkTagEffective(dropBoxEntryFileTag)) {
            val dropBoxEntryAnalysisServiceIntent =
                Intent(context, DropBoxEntryAnalysisService::class.java)
            dropBoxEntryAnalysisServiceIntent.putExtra(
                DropBoxManager.EXTRA_TAG,
                dropBoxEntryFileTag
            )
            dropBoxEntryAnalysisServiceIntent.putExtra(
                DropBoxManager.EXTRA_TIME,
                dropBoxEntryFileTime.toString()
            )
            val dropBoxEntryAnalysisServiceComponentName =
                ComponentName(context, DropBoxEntryAnalysisService::class.java)
            dropBoxEntryAnalysisServiceIntent.component = dropBoxEntryAnalysisServiceComponentName
            context.startService(dropBoxEntryAnalysisServiceIntent)
            Log.d(TAG, "onReceive: Start dropBoxEntryAnalysisService finish.")
        } else {
            Log.d(TAG, "onReceive: Not effective intent, miss it...")
        }
    }

    private fun checkTagEffective(tag: String): Boolean {
        return uploadTagList.contains(tag)
    }
}

DropBoxEntryAnalysisService.kt

/**
 * @ClassName: DropBoxEntryAnalysisService
 * @Desc: Service of DropBox file entry analysis.
 * */
class DropBoxEntryAnalysisService : IntentService("DropBoxEntryAnalysisService") {

    override fun onBind(intent: Intent): IBinder {
    }

    private fun uploadExceptionMsg(intent: Intent) {
        val dropBoxFileTag = intent.getStringExtra(DropBoxManager.EXTRA_TAG)
        val dropBoxFileTimestamp = intent.getStringExtra(DropBoxManager.EXTRA_TIME)

        if (dropBoxFileTag.isNullOrEmpty() || dropBoxFileTimestamp.isNullOrEmpty()) {
            return
        }

        val dropBoxFileTree: FileTreeWalk = File(DROPBOX_DIRECTORY).walk()
        dropBoxFileTree.maxDepth(1)
            .filter { it.isFile }
            .filter { it.canRead() }
            .filter {
                it.name.startsWith(dropBoxFileTag)
                it.name.contains(dropBoxFileTimestamp)
            }
            .iterator().forEach {
                val dropBoxParsedEntry = getDropBoxParsedEntryFromExpFile(it)
                uploadDropBoxEntry(dropBoxParsedEntry)
            }
    }
......
    private fun getDropBoxParsedEntryFromExpFile(exceptionFile: File): DropBoxParsedEntry {
        /** The file type must be one of 'txt' and 'gzip' **/
        return when (exceptionFile.extension) {
            "txt" -> {
                analysisExpTxtFileContent(exceptionFile)
            }
            "gz" -> {
                analysisGzipDropBoxFile(exceptionFile)
            }
            else -> DropBoxParsedEntry("")
        }
    }
    
    private fun uploadDropBoxEntry(dropBoxParsedEntry: DropBoxParsedEntry) {
        if (!dropBoxParsedEntry.isEmpty()) {
            val cStoreListener = CStoreUploader(applicationContext)
            cStoreListener.updateCStoreFile(dropBoxParsedEntry)
        } else {
            Log.e(TAG, "uploadDropBoxEntry: Get empty dropBoxParsedEntry!")
        }
    }
......
    private fun analysisExpTxtFileContent(exceptionTxtFile: File): DropBoxParsedEntry {
        ......
        return DropBoxParsedEntry("")
    }

    private fun analysisGzipDropBoxFile(exceptionGzipFile: File): DropBoxParsedEntry {
        ......
        return dropBoxParsedEntry
    }

    ......
    override fun onHandleIntent(intent: Intent?) {
        Log.d(TAG, "onHandleIntent: DropBoxEntryAnalysisService...")
        intent?.let { uploadExceptionMsg(it) }
    }

......
}

此处使用的文件上报是公司内部的接口，仅作为参考，也可以使用其他云平台进行上报，需要实现相应的接口：

CStoreUploader.kt

    private fun checkUploadEntryDuplicate(uploadEntry: String): Boolean {
        val uploadSharedPreferences =
            mContext.getSharedPreferences(UPLOAD_PREFERENCES, Context.MODE_PRIVATE)
        return uploadSharedPreferences.getBoolean(uploadEntry, false)
    }
    
    /**
     * @Fun: updateCStoreFile
     * @Des: Upload file .
     * **/
    fun updateCStoreFile(dropBoxParsedEntry: DropBoxParsedEntry) {
        if (checkUploadEntryDuplicate(dropBoxParsedEntry.dropBoxFileName)) {
            return
        }
        ......
        uploadDropBoxParsedEntry()
        ......
    }

实时性较低的异常上报

由于实时性较低，因此不需要频繁上报，只有重启开机或断网重连后再进行检测，并采用了sharePreferences进行了去重操作，避免上报的文件重复：

......
    private fun registerExceptionBootMsgUploader(context: Context) {
        val cm = context.getSystemService(Context.CONNECTIVITY_SERVICE) as ConnectivityManager
        val networkBuilder = NetworkRequest.Builder()

        networkBuilder.addTransportType(NetworkCapabilities.TRANSPORT_WIFI)
        networkBuilder.addTransportType(NetworkCapabilities.TRANSPORT_ETHERNET)
        cm.registerNetworkCallback(
            networkBuilder.build(),
            NetworkConnectionCallback(context)
        )
    }
......

NetworkConnectionCallback.kt

class NetworkConnectionCallback(context: Context) : ConnectivityManager.NetworkCallback() {
    private val mContext = context

    override fun onAvailable(network: Network) {
        Log.d(TAG, "Network connected, analysis boot reason.")
        uploadExceptionDropBoxParsedEntry()
    }

    override fun onLost(network: Network) {
        super.onLost(network)
        Log.d(TAG, "Network disconnect.")
    }

    private fun uploadExceptionDropBoxParsedEntry() {
        val dropBoxFileTree: FileTreeWalk = File(DROPBOX_DIRECTORY).walk()
        dropBoxFileTree.maxDepth(1)
            .filter { it.isFile }
            .filter { it.canRead() }
            .iterator().forEach {
                when {
                    it.name.contains(TAG_SYSTEM_LAST_KMSG) ->
                        uploadKernelExceptionDropBoxParsedEntry(it)
                    it.name.contains(TAG_SYSTEM_RECOVERY_LOG) ->
                        uploadRecoveryExceptionDropBoxParsedEntry(it)
                    it.name.contains(TAG_SYSTEM_FSCK) ->
                        uploadFileSystemExceptionDropBoxParsedEntry(it)
                    it.name.contains(TAG_SYSTEM_TOMBSTONE) ->
                        uploadSystemTombstoneExceptionDropBoxParsedEntry(it)
                    it.name.contains(TAG_SYSTEM_AUDIT) ->
                        uploadAuditExceptionDropBoxParsedEntry(it)
                }
            }

    }

    /** Check if has kernel exception, upload it.
     * */
    private fun uploadKernelExceptionDropBoxParsedEntry(kernelExpFile: File) {
        val contentLineList = kernelExpFile.readLines(Charset.forName("UTF-8"))
        if (contentLineList.size >= KERNEL_MSG_EXCEPTION_MIN_LINE_SIZE) {
            val dropBoxParsedEntry = DropBoxParsedEntry(kernelExpFile.name)
            ......
            cStoreListener.updateCStoreFile(dropBoxParsedEntry)
        }
    }


    /** Check if system boot into recovery mode, upload it.
     * */
    private fun uploadRecoveryExceptionDropBoxParsedEntry(recoveryExpFile: File) {
        val dropBoxParsedEntry = DropBoxParsedEntry(recoveryExpFile.name)
        ......
        cStoreListener.updateCStoreFile(dropBoxParsedEntry)
    }

    /** Check if file system has exception, upload it.
     * */
    private fun uploadFileSystemExceptionDropBoxParsedEntry(filesystemExpFile: File) {
        val dropBoxParsedEntry = DropBoxParsedEntry(filesystemExpFile.name)
        ......
        cStoreListener.updateCStoreFile(dropBoxParsedEntry)
    }

    /** Check if has system tombstone file, upload it.
     * */
    private fun uploadSystemTombstoneExceptionDropBoxParsedEntry(systemTombstoneExpFile: File) {
        val dropBoxParsedEntry = DropBoxParsedEntry(systemTombstoneExpFile.name)
        ......
        cStoreListener.updateCStoreFile(dropBoxParsedEntry)
    }

    /** Check if has system tombstone file, upload it.
     * */
    private fun uploadAuditExceptionDropBoxParsedEntry(auditExpFile: File) {
        val contentLineList = auditExpFile.readLines(Charset.forName("UTF-8"))
        if (contentLineList.size >= KERNEL_MSG_EXCEPTION_MIN_LINE_SIZE) {
            val dropBoxParsedEntry = DropBoxParsedEntry(auditExpFile.name)
            ......
            cStoreListener.updateCStoreFile(dropBoxParsedEntry)
        }
    }
......
}