Android 探索之 Task 分析(八)

本文将会分析如下几个task

transformResourcesWithMergeJavaResForRelease
transformClassesAndResourcesWithProguardForRelease
transformClassesWithMultidexlistForRelease
transformClassesWithDexForRelease
mergeReleaseJniLibFolders
transformNativeLibsWithMergeJniLibsForRelease
validateSigningRelease
packageRelease
assembleRelease

31.transformResourcesWithMergeJavaResForRelease

转换资源,合并lib resource资源,注意此处并不是合并java 而是 jar 包里携带的resource文件,也包含assets。
这个task就是将相应依赖的jar里面的resource合并到intermediates/transforms/mergeJavaRes/release目录。
此task的输出会是proguard task的输入之一
但是有一点需要注意,merge java resource时会有些文件时默认不会执行merge操作,也就是 packagingOptions 配置项中配置的内容,
packagingOption官方文档
默认不做merge的文件如下:

    @Inject
    public PackagingOptions() {
        // ATTENTION - keep this in sync with JavaDoc above.
        exclude("/META-INF/LICENSE");
        exclude("/META-INF/LICENSE.txt");
        exclude("/META-INF/MANIFEST.MF");
        exclude("/META-INF/NOTICE");
        exclude("/META-INF/NOTICE.txt");
        exclude("/META-INF/*.DSA");
        exclude("/META-INF/*.EC");
        exclude("/META-INF/*.SF");
        exclude("/META-INF/*.RSA");
        exclude("/META-INF/maven/**");
        exclude("/NOTICE");
        exclude("/NOTICE.txt");
        exclude("/LICENSE.txt");
        exclude("/LICENSE");


        // Exclude version control folders.
        exclude("**/.svn/**");
        exclude("**/CVS/**");
        exclude("**/SCCS/**");

        // Exclude hidden and backup files.
        exclude("**/.*/**");
        exclude("**/.*");
        exclude("**/*~");

        // Exclude index files
        exclude("**/thumbs.db");
        exclude("**/picasa.ini");

        // Exclude javadoc files
        exclude("**/about.html");
        exclude("**/package.html");
        exclude("**/overview.html");

        // Exclude stuff for unknown reasons
        exclude("**/_*");
        exclude("**/_*/**");

        // Merge services
        merge("/META-INF/services/**");
    }

源码分析

//com.android.build.gradle.internal.transforms.MergeJavaResourcesTransform
 @Override
    public void transform(@NonNull TransformInvocation invocation)
            throws IOException, TransformException {
        //1、创建release-mergeJavaRes/zip-cache 文件夹
        FileUtils.mkdirs(cacheDir);
        FileCacheByPath zipCache = new FileCacheByPath(cacheDir);

        TransformOutputProvider outputProvider = invocation.getOutputProvider();
        checkNotNull(outputProvider, "Missing output object for transform " + getName());

        //2、读取packaingOptions配置
        ParsedPackagingOptions packagingOptions = new ParsedPackagingOptions(this.packagingOptions);

        boolean full = false;
        IncrementalFileMergerState state = loadMergeState();
        if (state == null || !invocation.isIncremental()) {
            /*
             * This is a full build.
             */
            state = new IncrementalFileMergerState();
            outputProvider.deleteAll();
            full = true;
        }

        List<Runnable> cacheUpdates = new ArrayList<>();

        Map<IncrementalFileMergerInput, QualifiedContent> contentMap = new HashMap<>();
        List<IncrementalFileMergerInput> inputs =
                new ArrayList<>(
                        IncrementalFileMergerTransformUtils.toInput(
                                invocation,
                                zipCache,
                                cacheUpdates,
                                full,
                                contentMap));

        /*
         * In an ideal world, we could just send the inputs to the file merger. However, in the
         * real world we live in, things are more complicated :)
         *
         * We need to:
         *
         * 1. We need to bring inputs that refer to the project scope before the other inputs.
         *   项目的class和lib优先
         * 2. Prefix libraries that come from directories with "lib/".
         * 
         * 3. Filter all inputs to remove anything not accepted by acceptedPathsPredicate neither
         * by packagingOptions.
         */

        // Sort inputs to move project scopes to the start.
        inputs.sort((i0, i1) -> {
            int v0 = contentMap.get(i0).getScopes().contains(Scope.PROJECT)? 0 : 1;
            int v1 = contentMap.get(i1).getScopes().contains(Scope.PROJECT)? 0 : 1;
            return v0 - v1;
        });

        // Prefix libraries with "lib/" if we're doing libraries.
        assert mergedType.size() == 1;
        ContentType mergedType = this.mergedType.iterator().next();
        if (mergedType == ExtendedContentType.NATIVE_LIBS) {
            inputs =
                    inputs.stream()
                            .map(
                                    i -> {
                                        QualifiedContent qc = contentMap.get(i);
                                        if (qc.getFile().isDirectory()) {
                                            i =
                                                    new RenameIncrementalFileMergerInput(
                                                            i,
                                                            s -> "lib/" + s,
                                                            s -> s.substring("lib/".length()));
                                            contentMap.put(i, qc);
                                        }

                                        return i;
                                    })
                            .collect(Collectors.toList());
        }

        // 过滤到packagingOption中配置的exclude的 lib
        // Filter inputs.
        Predicate<String> inputFilter =
                acceptedPathsPredicate.and(
                        path -> packagingOptions.getAction(path) != PackagingFileAction.EXCLUDE);
        inputs = inputs.stream()
                .map(i -> {
                    IncrementalFileMergerInput i2 =
                            new FilterIncrementalFileMergerInput(i, inputFilter);
                    contentMap.put(i2, contentMap.get(i));
                    return i2;
                })
                .collect(Collectors.toList());

        /*
         * Create the algorithm used by the merge transform. This algorithm decides on which
         * algorithm to delegate to depending on the packaging option of the path. By default it
         * requires just one file (no merging).
         */
        // 根据配置的 exclude pick first merge none 等 配置不同的算法,做不同的merge
        StreamMergeAlgorithm mergeTransformAlgorithm = StreamMergeAlgorithms.select(path -> {
            PackagingFileAction packagingAction = packagingOptions.getAction(path);
            switch (packagingAction) {
                case EXCLUDE:
                    // Should have been excluded from the input.
                    throw new AssertionError();
                case PICK_FIRST:
                    return StreamMergeAlgorithms.pickFirst();
                case MERGE:
                    return StreamMergeAlgorithms.concat();
                case NONE:
                    return StreamMergeAlgorithms.acceptOnlyOne();
                default:
                    throw new AssertionError();
            }
        });

        /*
         * Create an output that uses the algorithm. This is not the final output because,
         * unfortunately, we still have the complexity of the project scope overriding other scopes
         * to solve.
         *
         * When resources inside a jar file are extracted to a directory, the results may not be
         * expected on Windows if the file names end with "." (bug 65337573), or if there is an
         * uppercase/lowercase conflict. To work around this issue, we copy these resources to a
         * jar file.
         */

        IncrementalFileMergerOutput baseOutput;
        if (mergedType == QualifiedContent.DefaultContentType.RESOURCES) {
            File outputLocation =
                    outputProvider.getContentLocation(
                            "resources", getOutputTypes(), getScopes(), Format.JAR);
            baseOutput =
                    IncrementalFileMergerOutputs.fromAlgorithmAndWriter(
                            mergeTransformAlgorithm, MergeOutputWriters.toZip(outputLocation));
        } else {
            File outputLocation =
                    outputProvider.getContentLocation(
                            "resources", getOutputTypes(), getScopes(), Format.DIRECTORY);
            baseOutput =
                    IncrementalFileMergerOutputs.fromAlgorithmAndWriter(
                            mergeTransformAlgorithm,
                            MergeOutputWriters.toDirectory(outputLocation));
        }

        /*
         * We need a custom output to handle the case in which the same path appears in multiple
         * inputs and the action is NONE, but only one input is actually PROJECT. In this specific
         * case we will ignore all other inputs.
         */
        // 特殊处理了 多个lib中都有同一个文件,但是其中有一个 是project依赖的,那么就会忽略掉其他的
        Set<IncrementalFileMergerInput> projectInputs =
                contentMap.keySet().stream()
                        .filter(i -> contentMap.get(i).getScopes().contains(Scope.PROJECT))
                        .collect(Collectors.toSet());

        IncrementalFileMergerOutput output = new DelegateIncrementalFileMergerOutput(baseOutput) {
            @Override
            public void create(
                    @NonNull String path,
                    @NonNull List<IncrementalFileMergerInput> inputs) {
                super.create(path, filter(path, inputs));
            }

            @Override
            public void update(
                    @NonNull String path,
                    @NonNull List<String> prevInputNames,
                    @NonNull List<IncrementalFileMergerInput> inputs) {
                super.update(path, prevInputNames, filter(path, inputs));
            }

            @Override
            public void remove(@NonNull String path) {
                super.remove(path);
            }

            @NonNull
            private ImmutableList<IncrementalFileMergerInput> filter(
                    @NonNull String path,
                    @NonNull List<IncrementalFileMergerInput> inputs) {
                PackagingFileAction packagingAction = packagingOptions.getAction(path);
                if (packagingAction == PackagingFileAction.NONE
                        && inputs.stream().anyMatch(projectInputs::contains)) {
                    inputs = inputs.stream()
                            .filter(projectInputs::contains)
                            .collect(ImmutableCollectors.toImmutableList());
                }

                return ImmutableList.copyOf(inputs);
            }
        };

        state = IncrementalFileMerger.merge(ImmutableList.copyOf(inputs), output, state);
        saveMergeState(state);

        cacheUpdates.forEach(Runnable::run);
    }

输入输出

-----task begin-------->
project:     project ':app_driver' 
name:        transformResourcesWithMergeJavaResForRelease 
group:       null 
description: null
conv:        [:]
inputs:
 /Users/dongkai/Code/XiWeiLogistics/ymm_app_driver_main_module/libs/yunceng.jar
 /Users/dongkai/.gradle/caches/transforms-1/files-1.1/activity_result_util-1.0.1.aar/4b6d13d6fe7210d069c29939ba8fea95/jars/classes.jar
...
...
...
 /Users/dongkai/Code/XiWeiLogistics/ymm_app_driver_main_module/build/intermediates/intermediate-jars/release/res.jar
 /Users/dongkai/Code/XiWeiLogistics/biz_common/build/intermediates/intermediate-jars/release/res.jar
outputs:
 /Users/dongkai/Code/XiWeiLogistics/app_driver/build/intermediates/incremental/release-mergeJavaRes/zip-cache
 /Users/dongkai/Code/XiWeiLogistics/app_driver/build/intermediates/transforms/mergeJavaRes/release
<------task end -------

从实际输出来看,只用到了/mergeJavaRes/release 目录下的产物,并未用到 release-mergeJavaRes/zip-cache 下的产物,应该就是缓存

32.transformClassesAndResourcesWithProguardForRelease

主要处理proguard,并生成proguard的产物

  • mapping.txt 就是proguard的mapping文件
  • dump.txt为描述apk文件中所有类文件间的内部结构
  • seeds.txt 列出了未被混淆的类和成员;
  • usage.txt 列出了从apk中删除的代码
    ProGuard 优化主要分为四个阶段:

Shrink , Optimize, Obfuscate , Preverify 四个阶段

  • Shrink: 删除没有被使用的类和方法。
  • Optimize: 对代码指令进行优化。
  • Obfuscate: 对代码名称进行混淆。
  • Preverify: 对 class 进行预校验,校验 StackMap /StackMapTable 属性。
    参考分析文档 Proguard 初探

源码分析

//com.android.build.gradle.internal.transforms.ProGuardTransform
    @Override
    public void transform(@NonNull final TransformInvocation invocation) throws TransformException {
        // only run one minification at a time (across projects)
        SettableFuture<TransformOutputProvider> resultFuture = SettableFuture.create();
        final Job<Void> job = new Job<>(getName(),
                new com.android.builder.tasks.Task<Void>() {
                    @Override
                    public void run(@NonNull Job<Void> job,
                            @NonNull JobContext<Void> context) throws IOException {
                        doMinification(
                                invocation.getInputs(),
                                invocation.getReferencedInputs(),
                                invocation.getOutputProvider());
                    }

                    @Override
                    public void finished() {
                        resultFuture.set(invocation.getOutputProvider());
                    }

                    @Override
                    public void error(Throwable e) {
                        resultFuture.setException(e);
                    }
                }, resultFuture);
        try {
            SimpleWorkQueue.push(job);

            // wait for the task completion.
            try {
                job.awaitRethrowExceptions();
            } catch (ExecutionException e) {
                throw new RuntimeException("Job failed, see logs for details", e.getCause());
            }
        } catch (InterruptedException e) {
            Thread.currentThread().interrupt();
            throw new RuntimeException(e);
        }
    }


private void doMinification(
            @NonNull Collection<TransformInput> inputs,
            @NonNull Collection<TransformInput> referencedInputs,
            @Nullable TransformOutputProvider output)
            throws IOException {
        try {
            checkNotNull(output, "Missing output object for transform " + getName());
            Set<ContentType> outputTypes = getOutputTypes();
            Set<? super Scope> scopes = getScopes();
            File outFile =
                    output.getContentLocation(
                            "combined_res_and_classes", outputTypes, scopes, Format.JAR);
            mkdirs(outFile.getParentFile());

            GlobalScope globalScope = variantScope.getGlobalScope();

            // set the mapping file if there is one.
            File testedMappingFile = computeMappingFile();
            if (testedMappingFile != null) {
                applyMapping(testedMappingFile);
            }

            // --- InJars / LibraryJars ---
            addInputsToConfiguration(inputs, false);
            addInputsToConfiguration(referencedInputs, true);

            // libraryJars: the runtime jars, with all optional libraries.
            for (File runtimeJar : globalScope.getAndroidBuilder().getBootClasspath(true)) {
                libraryJar(runtimeJar);
            }

            // --- Out files ---
            // 配置输出文件为/transforms/proguard/release/0.jar
            outJar(outFile);

            // proguard doesn't verify that the seed/mapping/usage folders exist and will fail
            // if they don't so create them.
            mkdirs(proguardOut);

            for (File configFile : getAllConfigurationFiles()) {
                LOG.info("Applying ProGuard configuration file {}", configFile);
                applyConfigurationFile(configFile);
            }

            configuration.printMapping = printMapping;
            configuration.dump = dump;
            configuration.printSeeds = printSeeds;
            configuration.printUsage = printUsage;

            forceprocessing();
            // 真正proguard的代码逻辑
            runProguard();
        } catch (Exception e) {
            if (e instanceof IOException) {
                throw (IOException) e;
            }

            throw new IOException(e);
        }
    }

Proguard 实际执行的代码为:

new ProGuard(configuration).execute();

进入源码分析:

// proguard.ProGuard
/**
     * Performs all subsequent ProGuard operations.
     */
    public void execute() throws IOException
    {
        System.out.println(VERSION);
        // GPL 许可证检查

        GPL.check();

        if (configuration.printConfiguration != null)
        {
            printConfiguration();
        }
        // 检查混淆配置是否符合规则并正确
        new ConfigurationChecker(configuration).check();

        if (configuration.programJars != null     &&
            configuration.programJars.hasOutput() &&
            new UpToDateChecker(configuration).check())
        {
            return;
        }
       // 读取所有的jar包中的类到类池中,重要的入口,后续所有的操作都需要访问类池
       // 在此会生成两个类池,一个是 libraryClassPool ,一个是 programClassPool 
       // libraryjars 描述程序运行中需要用的环境, 主要是为后面优化阶段提供信息分析。libraryjars 一般情况为 JRE 下的 rt.jar 和一些特定平台类型的 jar 。
       // programeclass 为 项目的 class
        readInput();

        if (configuration.shrink    ||
            configuration.optimize  ||
            configuration.obfuscate ||
            configuration.preverify)
        { 
            // 从程序类中清除所有的JSE预验证信息
            // 实际是清除StackMapTable属性,在java 6 版本中JVM在class文件中引入了栈图 StackMapTable属性,作用是为了提交JVM在类型检查的验证过程的效率,在字节码的code属性中最多包含一个StackMapTable属性
            clearPreverification();
        }

        if (configuration.printSeeds != null ||
            configuration.shrink    ||
            configuration.optimize  ||
            configuration.obfuscate ||
            configuration.preverify)
        {
            /** 基于两个 ClassPool 对所有的 Class 进行连接.
             *   1.连接包括所有的类的层级关系 ( 父类,子类,interface )。
             *   2.连接注解中 enum 常量。
             *   3.连接 code 字节码相关字段和相关类。
             *   method 的操作:关联对应的 class 和 method。
             *   field 的 操作:关联对应的 class 和 field。
             *   3.连接反射信息,所有满足如下规则的反射代码会有正确的 proguard,但并不是所有的都生效
             *   反射是根据类名或方法名或字段名进行操作的。当我们将反射使用的字符串跟对应的类或方法或字段连接上. 当对应的类或方法或字段混淆的时候同步变更,那么反射依旧生效, 之所以出现了 NoSuchMethodException, * NoSuchFieldException,ClassNotFoundException 等问题,就是因为不同步更改信息. 同步更改需要在Initialize 阶段将反射信息连接上对应的类字段方法. 这里的连接并不是没有缺陷的.但是会处理以下几种情况
             *   Class.forName("SomeClass");
             *   Class.forName("SomeClass").newInstance().
             *   AtomicIntegerFieldUpdater.newUpdater(A.class, "someField")
             *   AtomicLongFieldUpdater.newUpdater(A.class, "someField")
             *   AtomicReferenceFieldUpdater.newUpdater(A.class, B.class,"someField")
             *   AtomicIntegerFieldUpdater.newUpdater(..., "someField")
             *   AtomicLongFieldUpdater.newUpdater(..., "someField")
             *   AtomicReferenceFieldUpdater.newUpdater(..., "someField")
             *   SomeClass.class.getMethod("someMethod",...)
             *   SomeClass.class.getDeclaredMethod("someMethod",...)
             *   SomeClass.class.getField("someMethod",...)
             *   SomeClass.class.getDeclaredFields("someMethod",...)
             *   SomeClass.class.getConstructor("someMethod",...)
             *   SomeClass.class.getDeclaredConstructor("someMethod",...)
             *   这里情况,反射信息能被正确连接.
             **/

            initialize();
        }

        if (configuration.targetClassVersion != 0)
        {
            //android没有用到,暂不分析
            target();
        }

        if (configuration.printSeeds != null)
        {
            // 把keep匹配的类和方法输出到文件中,可以用来验证自己设定的规则是否生效.
            printSeeds();
        }

        if (configuration.shrink)
        {
            // 根据 Configuration Roots 开始标记, 同时根据 Roots 为入口开始发散 . 标记完成以后, 删除未被标记的类或成员. 最终得到的是精简的 ClassPool 。
            shrink();
        }

        if (configuration.preverify)
        {
            //进行预检测,然后子程序一致化,android没有用到,暂不分析
            inlineSubroutines();
        }
        // 代码指令优化
        // Optimize 是四个阶段最为复杂的地方。也是耗时最长的阶段。
        // Optimize 会在该阶段通过对 代码指令、 堆栈, 局部变量以及数据流分析.来模拟程序运行中尽可能出现的情况来优化和简化代码. 为了数据流分析的需要 Optimize 会多次遍历所有字节码。ProGuard 会开启多线程来加快速度。

        if (configuration.optimize)
        {
            for (int optimizationPass = 0;
                 optimizationPass < configuration.optimizationPasses;
                 optimizationPass++)
            {
                if (!optimize())
                {
                    // Stop optimizing if the code doesn't improve any further.
                    break;
                }

                // Shrink again, if we may.
                if (configuration.shrink)
                {
                    // Don't print any usage this time around.
                    configuration.printUsage       = null;
                    configuration.whyAreYouKeeping = null;
                    // 二次shrink,优化后再次shrink
                    shrink();
                }
            }
        }

        if (configuration.optimize)
        {
            //在方法内联和类合并等优化之后,消除所有程序类的行号
            linearizeLineNumbers();
        }

        if (configuration.obfuscate)
        {
           //执行混淆处理步骤。 默认开启,否则用啥混淆
           //将类,字段,方法的名称简化成短名字, 简化需要依据 java 的规范, 方法名应符合定义没有非法字符. 虚方法在 class 继承中方法名称保持一致. 同个范围内字段或方法描述符,签名相同的时候名称唯一, 相同包下 class 名称唯一. 从 library 中继承的方法名称不变等等。

            obfuscate();
        }

        if (configuration.optimize)
        {
            //混淆完成再次进行优化,合并行号
            trimLineNumbers();
        }

        if (configuration.preverify)
        {
            //执行预验证步骤 看混淆后的class能不能正常运行
            preverify();
        }

        if (configuration.shrink    ||
            configuration.optimize  ||
            configuration.obfuscate ||
            configuration.preverify)
        {
            //对所有程序类的元素排序。
            sortClassElements();
        }

        if (configuration.programJars.hasOutput())
        {      
            // 写入输出类文件。
            writeOutput();
        }

        if (configuration.dump != null)
        {
            //打印出程序类的内容。
            dump();
        }

输入输出

-----task begin-------->
project:     project ':app_driver' 
name:        transformClassesAndResourcesWithProguardForRelease 
group:       null 
description: null
conv:        [:]
inputs:
 /Users/dongkai/Code/XiWeiLogistics/app_driver/proguard-project.txt
 /Users/dongkai/Code/XiWeiLogistics/build/intermediates/proguard-files/proguard-android.txt-3.1.4
...
...
...
 /Users/dongkai/.gradle/caches/transforms-1/files-1.1/sdk-2.13.3.0.aar/cbde3b805a388d5cd0d41d9ac1be0c66/proguard.txt
 /Users/dongkai/.gradle/caches/transforms-1/files-1.1/vivo-1.0.4.aar/8865158a811f23e9d9b6545f5ff48d01/proguard.txt
 /Users/dongkai/.gradle/caches/transforms-1/files-1.1/mmkv-static-1.0.17.aar/0c39c42d42d3b884ed1b1cdbff217b58/proguard.txt
 /Users/dongkai/Code/XiWeiLogistics/app_driver/build/intermediates/transforms/desugar/release/212.jar
 ...
 ...
 ...
 /Users/dongkai/Code/XiWeiLogistics/app_driver/build/intermediates/transforms/desugar/release/69.jar
 /Users/dongkai/Code/XiWeiLogistics/app_driver/build/intermediates/transforms/desugar/release/2/pl/droidsonroids/gif/R.class
 /Users/dongkai/Code/XiWeiLogistics/app_driver/build/intermediates/transforms/desugar/release/2/pl/droidsonroids/gif/R$styleable.class
 /Users/dongkai/Code/XiWeiLogistics/app_driver/build/intermediates/transforms/desugar/release/2/pl/droidsonroids/gif/R$attr.class
...
...
...
 /Users/dongkai/Code/XiWeiLogistics/app_driver/build/intermediates/transforms/mergeJavaRes/release/__content__.json
 /Users/dongkai/.gradle/caches/modules-2/files-2.1/com.ymm.lib/lib_eversocket/2.2.3/1e1438faf1589ce0082832a759fadc4a3021919d/lib_eversocket-2.2.3.jar
outputs:
 /Users/dongkai/Code/XiWeiLogistics/app_driver/build/outputs/mapping/release/mapping.txt
 /Users/dongkai/Code/XiWeiLogistics/app_driver/build/outputs/mapping/release/dump.txt
 /Users/dongkai/Code/XiWeiLogistics/app_driver/build/outputs/mapping/release/seeds.txt
 /Users/dongkai/Code/XiWeiLogistics/app_driver/build/outputs/mapping/release/usage.txt
 /Users/dongkai/Code/XiWeiLogistics/app_driver/build/intermediates/transforms/proguard/release
<------task end -------

输出为四个txt和一个混淆后的jar包

33.transformClassesWithMultidexlistForRelease

该task其实并没有做特别的事情,只是输出了三个文件,为后续打dex分包做准备

  • componentClasses.jar 经过shrinkWithProguard得到(componentClasses.jar,但是在output路径中并未出现)
  • components.flags
  • maindexlist.txt ----> 通过一些列操作,计算出来的一个列表,记录放入主dex中的所有class

源码分析

//com.android.build.gradle.internal.transforms.MultiDexTransform
@Override
    public void transform(@NonNull TransformInvocation invocation)
            throws IOException, TransformException, InterruptedException {
        // Re-direct the output to appropriate log levels, just like the official ProGuard task.
        LoggingManager loggingManager = invocation.getContext().getLogging();
        loggingManager.captureStandardOutput(LogLevel.INFO);
        loggingManager.captureStandardError(LogLevel.WARN);

        try {
            Map<MainDexListTransform.ProguardInput, Set<File>> inputs =
                    MainDexListTransform.getByInputType(invocation);
            File input =
                    Iterables.getOnlyElement(
                            inputs.get(MainDexListTransform.ProguardInput.INPUT_JAR));
           // 生成componentClasses.jar
            shrinkWithProguard(input, inputs.get(MainDexListTransform.ProguardInput.LIBRARY_JAR));
            computeList(input);
        } catch (ParseException | ProcessException e) {
            throw new TransformException(e);
        }
    }

// computeList
private void computeList(File _allClassesJarFile) throws ProcessException, IOException {
        // manifest components plus immediate dependencies must be in the main dex.
        Set<String> mainDexClasses = callDx(
                _allClassesJarFile,
                variantScope.getProguardComponentsJarFile());

        if (userMainDexKeepFile != null) {
            mainDexClasses = ImmutableSet.<String>builder()
                    .addAll(mainDexClasses)
                    .addAll(Files.readLines(userMainDexKeepFile, Charsets.UTF_8))
                    .build();
        }

        String fileContent = Joiner.on(System.getProperty("line.separator")).join(mainDexClasses);
        // 生成 manifest_keep.txt
        Files.write(fileContent, mainDexListFile, Charsets.UTF_8);

    }
// callDx
    private Set<String> callDx(File allClassesJarFile, File jarOfRoots) throws ProcessException {
        EnumSet<AndroidBuilder.MainDexListOption> mainDexListOptions =
                EnumSet.noneOf(AndroidBuilder.MainDexListOption.class);
        if (!keepRuntimeAnnotatedClasses) {
            mainDexListOptions.add(
                    AndroidBuilder.MainDexListOption.DISABLE_ANNOTATION_RESOLUTION_WORKAROUND);
            Logging.getLogger(MultiDexTransform.class).warn(
                    "Not including classes with runtime retention annotations in the main dex.\n"
                            + "This can cause issues with reflection in older platforms.");
        }

        return variantScope.getGlobalScope().getAndroidBuilder().createMainDexList(
                allClassesJarFile, jarOfRoots, mainDexListOptions);
    }
 //继续调用 createMainDexList 方法创建maindex
 // jarOfRoots 为上一步中生成的componentClasses.jar
 public Set<String> createMainDexList(
            @NonNull File allClassesJarFile,
            @NonNull File jarOfRoots,
            @NonNull EnumSet<MainDexListOption> options) throws ProcessException {

        BuildToolInfo buildToolInfo = mTargetInfo.getBuildTools();
        ProcessInfoBuilder builder = new ProcessInfoBuilder();

        String dx = buildToolInfo.getPath(BuildToolInfo.PathId.DX_JAR);
        if (dx == null || !new File(dx).isFile()) {
            throw new IllegalStateException("dx.jar is missing");
        }

        builder.setClasspath(dx);
        builder.setMain("com.android.multidex.ClassReferenceListBuilder");

        if (options.contains(MainDexListOption.DISABLE_ANNOTATION_RESOLUTION_WORKAROUND)) {
            builder.addArgs("--disable-annotation-resolution-workaround");
        }

        builder.addArgs(jarOfRoots.getAbsolutePath());
        builder.addArgs(allClassesJarFile.getAbsolutePath());

        CachedProcessOutputHandler processOutputHandler = new CachedProcessOutputHandler();

        mJavaProcessExecutor.execute(builder.createJavaProcess(), processOutputHandler)
                .rethrowFailure()
                .assertNormalExitValue();

        LineCollector lineCollector = new LineCollector();
        processOutputHandler.getProcessOutput().processStandardOutputLines(lineCollector);
        return ImmutableSet.copyOf(lineCollector.getResult());
    }
// 执行createMainDexList
public Set<String> createMainDexList(
            @NonNull File allClassesJarFile,
            @NonNull File jarOfRoots,
            @NonNull EnumSet<MainDexListOption> options) throws ProcessException {

        BuildToolInfo buildToolInfo = mTargetInfo.getBuildTools();
        ProcessInfoBuilder builder = new ProcessInfoBuilder();

        String dx = buildToolInfo.getPath(BuildToolInfo.PathId.DX_JAR);
        if (dx == null || !new File(dx).isFile()) {
            throw new IllegalStateException("dx.jar is missing");
        }

        builder.setClasspath(dx);
        builder.setMain("com.android.multidex.ClassReferenceListBuilder");

        if (options.contains(MainDexListOption.DISABLE_ANNOTATION_RESOLUTION_WORKAROUND)) {
            builder.addArgs("--disable-annotation-resolution-workaround");
        }

        builder.addArgs(jarOfRoots.getAbsolutePath());
        builder.addArgs(allClassesJarFile.getAbsolutePath());

        CachedProcessOutputHandler processOutputHandler = new CachedProcessOutputHandler();

        mJavaProcessExecutor.execute(builder.createJavaProcess(), processOutputHandler)
                .rethrowFailure()
                .assertNormalExitValue();

        LineCollector lineCollector = new LineCollector();
        processOutputHandler.getProcessOutput().processStandardOutputLines(lineCollector);
        return ImmutableSet.copyOf(lineCollector.getResult());
    }
//mJavaProcessExecutor.execute最终会调用project.javaexec执行一个外部的java进程(MainDexListBuilder)
//MainDexListBuilder的构造函数中拿到传入rootJar和pathString,然后构造了mainListBuilder
//主要是调用了 mainListBuilder.addRoots(jarOfRoots);

主要逻辑代码为computeList
MainDexListTransform.getByInputType(invocation)先将input中的DirectoryInputs和jarInput组合成单一集合,然后根据input的scope类型是否为PROVIDED_ONLY分离成ProguardInput.INPUT_JAR 和 ProguardInput.LIBRARY_JAR和对应的files存进map中。
shrinkWithProguard 生成 componentClasses.jar
computeList()生成了maindexlist.txt,并且是以app模块里的类加上它所依赖的类进行maindex的划分 。

输入输出

-----task begin-------->
project:     project ':app_driver' 
name:        transformClassesWithMultidexlistForRelease 
group:       null 
description: null
conv:        [:]
inputs:
 /Users/dongkai/Code/XiWeiLogistics/app_driver/build/intermediates/multi-dex/release/manifest_keep.txt
 /Users/dongkai/.gradle/caches/modules-2/files-2.1/com.ymm.lib/lib_eversocket/2.2.3/1e1438faf1589ce0082832a759fadc4a3021919d/lib_eversocket-2.2.3.jar
 /Users/dongkai/Code/XiWeiLogistics/app_driver/build/intermediates/transforms/proguard/release/0.jar
 /Users/dongkai/Code/XiWeiLogistics/app_driver/build/intermediates/transforms/proguard/release/__content__.json
outputs:
 /Users/dongkai/Code/XiWeiLogistics/app_driver/build/intermediates/multi-dex/release/maindexlist.txt
 /Users/dongkai/Code/XiWeiLogistics/app_driver/build/intermediates/multi-dex/release/components.flags
<------task end -------

34.transformClassesWithDexForRelease

将 maindexlist.txt 和 混淆过的jar包 处理成multidex,具体流程就是分包过程,
分包过程中如果有配置 dexOptions则会影响分包的过程,dexOption示例如下:

    dexOptions {
        javaMaxHeapSize "1g"
        preDexLibraries = false
        additionalParameters = [    //配置multidex参数
                                '--multi-dex',//多dex分包
                                '--set-max-idx-number=30000',//每个包内方法数上限
                                '--main-dex-list='+projectDir+'/main-dex-rule', //打包到主classes.dex的文件列表,默认使用maindextlist.txt
                                '--minimal-main-dex'
        ]
    }

主要流程:
1.加载proguard的产物得到所有的class
2.加载maindexlist,根据maindexlist对proguard后的所有class进行拆分,将主、从dex的class文件分开
3.先打主dex,然后打出从dex并输出到固定目录

源码分析

//com.android.build.gradle.internal.transforms.DexTransform
//调用链为:
DexTransform.transform() 
-> DexByteCodeConverter.convertByteCode()  maindexlist和proguard产物还有dexOption
-> DexProcessBuilder.runDexer() 
-> DexProcessBuilder.dexInProcess() 
-> DexWrapper.run()  
-> DexWrapper.buildArguments() // 返回 dex打包Result
-> new Main(dxContext).runDx(args) // com.android.dx.command.dexer.Main
-> Main.runDx()
-> Main.runMultiDex() // 此处会读取 maindexList列表 为后续做filter使用
-> Main.processAllFiles()

看下实际的multidex打包过程

// with --main-dex-list
                FileNameFilter mainPassFilter = args.strictNameCheck ? new MainDexListFilter() :
                    new BestEffortMainDexListFilter();
                   
                // forced in main dex
                // 将特定的class加入到 maindex 的 container
                for (int i = 0; i < fileNames.length; i++) {
                    processOne(fileNames[i], mainPassFilter);
                }

                if (dexOutputFutures.size() > 0) {
                    throw new DexException("Too many classes in " + Arguments.MAIN_DEX_LIST_OPTION
                            + ", main dex capacity exceeded");
                }

                if (args.minimalMainDex) {
                    // start second pass directly in a secondary dex file.

                    // Wait for classes in progress to complete
                    synchronized(dexRotationLock) {
                        while(maxMethodIdsInProcess > 0 || maxFieldIdsInProcess > 0) {
                            try {
                                dexRotationLock.wait();
                            } catch(InterruptedException ex) {
                                /* ignore */
                            }
                        }
                    }

                    rotateDexFile();
                }

                // remaining files 处理除 maindex 以外的class 做集合
                FileNameFilter filter = new RemoveModuleInfoFilter(new NotFilter(mainPassFilter));
                for (int i = 0; i < fileNames.length; i++) {
                    processOne(fileNames[i], filter);
                }

输入输出

-----task begin-------->
project:     project ':app_driver' 
name:        transformClassesWithDexForRelease 
group:       null 
description: null
conv:        [:]
inputs:
 /Users/dongkai/Code/XiWeiLogistics/app_driver/build/intermediates/multi-dex/release/maindexlist.txt
 /Users/dongkai/Code/XiWeiLogistics/app_driver/build/intermediates/transforms/proguard/release/0.jar
outputs:
 /Users/dongkai/Code/XiWeiLogistics/app_driver/build/intermediates/transforms/dex/release
<------task end -------

35.mergeReleaseJniLibFolders

与mergeReleaseAssets task 很相似,就是将相应的目录下的so文件merge到一个指定的目录下。

源码分析

//com.android.build.gradle.tasks.MergeSourceSetFolders
 @Override
    protected void doFullTaskAction() throws IOException {
        // this is full run, clean the previous output
        File destinationDir = getOutputDir();
        FileUtils.cleanOutputDir(destinationDir);

        List<AssetSet> assetSets = computeAssetSetList();

        // create a new merger and populate it with the sets.
        AssetMerger merger = new AssetMerger();

        try {
            for (AssetSet assetSet : assetSets) {
                // set needs to be loaded.
                assetSet.loadFromFiles(getILogger());
                merger.addDataSet(assetSet);
            }

            // get the merged set and write it down.
            MergedAssetWriter writer = new MergedAssetWriter(destinationDir, workerExecutor);

            merger.mergeData(writer, false /*doCleanUp*/);

            // No exception? Write the known state.
            merger.writeBlobTo(getIncrementalFolder(), writer, false);
        } catch (MergingException e) {
            getLogger().error("Could not merge source set folders: ", e);
            merger.cleanBlob(getIncrementalFolder());
            throw new ResourceException(e.getMessage(), e);
        }
    }

输入输出

-----task begin-------->
project:     project ':app_driver' 
name:        mergeReleaseJniLibFolders 
group:       null 
description: null
conv:        [:]
inputs:
 /Users/dongkai/Code/XiWeiLogistics/app_driver/libs
 /Users/dongkai/Code/XiWeiLogistics/app_driver/src/release/jniLibs
outputs:
 /Users/dongkai/Code/XiWeiLogistics/app_driver/build/intermediates/incremental/mergeReleaseJniLibFolders
 /Users/dongkai/Code/XiWeiLogistics/app_driver/build/intermediates/jniLibs/release
<------task end -------

36.transformNativeLibsWithMergeJniLibsForRelease

源码分析

// 与 transformResourcesWithMergeJavaResForRelease 的源码一致

输入输出

-----task begin-------->
project:     project ':app_driver' 
name:        transformNativeLibsWithMergeJniLibsForRelease 
group:       null 
description: null
conv:        [:]
inputs:
 /Users/dongkai/Code/XiWeiLogistics/ymm_app_driver_main_module/libs/yunceng.jar
 /Users/dongkai/.gradle/caches/transforms-1/files-1.1/activity_result_util-1.0.1.aar/4b6d13d6fe7210d069c29939ba8fea95/jars/classes.jar
 /Users/dongkai/.gradle/caches/transforms-1/files-1.1/biz_kefu-1.0.2.aar/ee1777acecc24eeb4e8b78c1aae4dd65/jars/classes.jar
 /Users/dongkai/.gradle/caches/transforms-1/files-1.1/biz_kefu_service-1.0.1.aar/c04a2060a018ecf52b962c7ce4ef6a71/jars/classes.jar
 /Users/dongkai/.gradle/caches/transforms-1/files-1.1/lib-inbox-1.8.1.aar/5193a8ebe7782cafd81f065f97375fb3/jars/classes.jar
 ...
 ...
 ...
 /Users/dongkai/.gradle/caches/transforms-1/files-1.1/mmkv-static-1.0.17.aar/0c39c42d42d3b884ed1b1cdbff217b58/jars/classes.jar
 /Users/dongkai/.gradle/caches/transforms-1/files-1.1/voice_impl-1.1.0.aar/04d9d4ac1edd9169813ef3c22774bbc6/jni/mips/libmsc.so
 /Users/dongkai/.gradle/caches/transforms-1/files-1.1/voice_impl-1.1.0.aar/04d9d4ac1edd9169813ef3c22774bbc6/jni/armeabi-v7a/libmsc.so
 ...
 ...
 ...
 /Users/dongkai/.gradle/caches/transforms-1/files-1.1/mmkv-static-1.0.17.aar/0c39c42d42d3b884ed1b1cdbff217b58/jni/x86_64/libmmkv.so
 /Users/dongkai/Code/XiWeiLogistics/ymm_app_driver_main_module/build/intermediates/intermediate-jars/release/res.jar
 /Users/dongkai/Code/XiWeiLogistics/biz_common/build/intermediates/intermediate-jars/release/res.jar
 /Users/dongkai/Code/XiWeiLogistics/ymm_app_driver_main_module/build/intermediates/intermediate-jars/release/jni/armeabi-v7a/libyunceng.so
 /Users/dongkai/Code/XiWeiLogistics/ymm_app_driver_main_module/build/intermediates/intermediate-jars/release/jni/armeabi/libyunceng.so
 /Users/dongkai/Code/XiWeiLogistics/app_driver/build/intermediates/jniLibs/release/armeabi/libgnustl_shared.so
outputs:
 /Users/dongkai/Code/XiWeiLogistics/app_driver/build/intermediates/incremental/release-mergeJniLibs/zip-cache
 /Users/dongkai/Code/XiWeiLogistics/app_driver/build/intermediates/transforms/mergeJniLibs/release
<------task end -------

37.validateSigningRelease

这个task就是在没有keystore时创建一个keystore,也就是在debug的时候,会使用默认的keystore,这也是为什么在debug模式下,没有配置keystore时,apk仍然能够安装使用。

源码分析

//com.android.build.gradle.internal.tasks.ValidateSigningTask
// 看下注释吧
/**
 * A Gradle Task to check that the keystore file is present for this variant's signing config.
 *
 * If the keystore is the default debug keystore, it will be created if it is missing.
 *
 * This task has no explicit inputs, but is forced to run if the signing config keystore file is
 * not present.
 */

输入输出

-----task begin-------->
project:     project ':app_driver' 
name:        validateSigningRelease 
group:       null 
description: null
conv:        [:]
inputs:
outputs:
 /Users/dongkai/Code/XiWeiLogistics/app_driver/build/intermediates/incremental/validateSigningRelease
<------task end -------

38.packageRelease

源码分析

//com.android.build.gradle.tasks.PackageApplication 继承自 PackageAndroidArtifact
// 如果有splitfile 则会循环调用如下方法,进行产出apk
private void doTask(
            @NonNull ApkInfo apkData,
            @NonNull File incrementalDirForSplit,
            @NonNull File outputFile,
            @NonNull FileCacheByPath cacheByPath,
            @NonNull BuildElements manifestOutputs,
            @NonNull ImmutableMap<RelativeFile, FileStatus> changedDex,
            @NonNull ImmutableMap<RelativeFile, FileStatus> changedJavaResources,
            @NonNull ImmutableMap<RelativeFile, FileStatus> changedAssets,
            @NonNull ImmutableMap<RelativeFile, FileStatus> changedAndroidResources,
            @NonNull ImmutableMap<RelativeFile, FileStatus> changedNLibs)
            throws IOException {

        ImmutableMap.Builder<RelativeFile, FileStatus> javaResourcesForApk =
                ImmutableMap.builder();
        javaResourcesForApk.putAll(changedJavaResources);
        // 处理instant run
        if (isInInstantRunMode()) {
            changedDex = ImmutableMap.copyOf(
                    Maps.filterKeys(
                            changedDex,
                            Predicates.compose(
                                    Predicates.in(getDexFolders().getFiles()),
                                    RelativeFile::getBase
                            )));
        }
        final ImmutableMap<RelativeFile, FileStatus> dexFilesToPackage = changedDex;
        //  abi filter
        String filter = null;
        FilterData abiFilter = apkData.getFilter(OutputFile.FilterType.ABI);
        if (abiFilter != null) {
            filter = abiFilter.getIdentifier();
        }
        // 根据apk data 获取 manifest
        // find the manifest file for this split.
        BuildOutput manifestForSplit = manifestOutputs.element(apkData);

        if (manifestForSplit == null) {
            throw new RuntimeException(
                    "Found a .ap_ for split "
                            + apkData
                            + " but no "
                            + manifestType
                            + " associated manifest file");
        }
        // 创建output目录
        FileUtils.mkdirs(outputFile.getParentFile());
        //创建IncrementalPackager 对象packager,然后对dex、javares、assets、AndroidResources、NativeLib执行更新操作,也就是写入操作。执行这些update操作,最终都是调用了IncrementalPackager种的update()方法。
        try (IncrementalPackager packager =
                new IncrementalPackagerBuilder()
                        .withOutputFile(outputFile)
                        .withSigning(signingConfig)
                        .withCreatedBy(getBuilder().getCreatedBy())
                        .withMinSdk(getMinSdkVersion())
                        // TODO: allow extra metadata to be saved in the split scope to avoid
                        // reparsing
                        // these manifest files.
                        .withNativeLibraryPackagingMode(
                                PackagingUtils.getNativeLibrariesLibrariesPackagingMode(
                                        manifestForSplit.getOutputFile()))
                        .withNoCompressPredicate(
                                PackagingUtils.getNoCompressPredicate(
                                        aaptOptionsNoCompress, manifestForSplit.getOutputFile()))
                        .withIntermediateDir(incrementalDirForSplit)
                        .withProject(getProject())
                        .withDebuggableBuild(getDebugBuild())
                        .withAcceptedAbis(filter == null ? abiFilters : ImmutableSet.of(filter))
                        .withJniDebuggableBuild(getJniDebugBuild())
                        .build()) {
            packager.updateDex(dexFilesToPackage);
            packager.updateJavaResources(changedJavaResources);
            packager.updateAssets(changedAssets);
            packager.updateAndroidResources(changedAndroidResources);
            packager.updateNativeLibraries(changedNLibs);
            // Only report APK as built if it has actually changed.
            if (packager.hasPendingChangesWithWait()) {
                // FIX-ME : below would not work in multi apk situations. There is code somewhere
                // to ensure we only build ONE multi APK for the target device, make sure it is still
                // active.
                instantRunContext.addChangedFile(instantRunFileType, outputFile);
            }
        }

        /*
         * Save all used zips in the cache.
         */
        Stream.concat(
                        dexFilesToPackage.keySet().stream(),
                        Stream.concat(
                                changedJavaResources.keySet().stream(),
                                Stream.concat(
                                        changedAndroidResources.keySet().stream(),
                                        changedNLibs.keySet().stream())))
                .map(RelativeFile::getBase)
                .filter(File::isFile)
                .distinct()
                .forEach(
                        (File f) -> {
                            try {
                                cacheByPath.add(f);
                            } catch (IOException e) {
                                throw new IOExceptionWrapper(e);
                            }
                        });
    }

输入输出

-----task begin-------->
project:     project ':app_driver' 
name:        packageRelease 
group:       null 
description: null
conv:        [:]
inputs:
 /Users/dongkai/Code/XiWeiLogistics/app_driver/build/interme diates/splits-support/release/apk-list/apk-list.gson
 /Users/dongkai/Code/XiWeiLogistics/app_driver/build/intermediates/assets/release
 /Users/dongkai/Code/XiWeiLogistics/app_driver/build/intermediates/transforms/dex/release/0
 /Users/dongkai/Code/XiWeiLogistics/app_driver/build/intermediates/transforms/proguard/release/0.jar
 /Users/dongkai/Code/XiWeiLogistics/app_driver/build/intermediates/transforms/mergeJniLibs/release/0
 /Users/dongkai/Code/XiWeiLogistics/app_driver/build/intermediates/manifests/full/release
 /Users/dongkai/Code/XiWeiLogistics/app_driver/build/intermediates/res/release
 /Users/dongkai/Code/XiWeiLogistics/XiWeiLogistics.keystore
outputs:
 /Users/dongkai/Code/XiWeiLogistics/app_driver/build/intermediates/incremental/packageRelease/tmp
 /Users/dongkai/Code/XiWeiLogistics/app_driver/build/outputs/apk/release
<------task end -------

分析下输入目录:

  • splits-support/release/apk-list/apk-list.gson split
  • assets/release assets目录下的文件;
  • transforms/dex/release/0 dex 文件;
  • /transforms/proguard/release/0.jar proguard 产物
  • transforms/mergeJniLibs/release/0 so文件;
  • manifests/full/release manifest文件;
  • res/release resources 文件;
  • XiWeiLogistics.keystore keystore 文件;

输出文件即为apk文件。

39.assembleRelease

锚点task,为了激活所有的依赖task

源码分析

// 普通的task,
group 为 BUILD_GROUP

输入输出

-----task begin-------->
project:     project ':app_driver' 
name:        assembleRelease 
group:       build 
description: Assembles all Release builds.
conv:        [:]
inputs:
outputs:
<------task end -------

这年头生活不易,人生不息,折腾不止!!!!

发表评论