Jenkins:批量自动将 Maven 类型 Job 迁移到自由风格类型

为什么要迁移?

近期进行 Jenkins 从1.X到2.X的升级演练
Jenkins2 最新版本只能在 JDK8 或 JDK11 版本下运行,我所使用的 JDK 版本为 JDK8
在构建 Maven Job,Job 配置的 JDK 版本为 JDK7时,构建报错

$ /usr/local/java/bin/java -cp /data/jenkins/maven31-agent.jar:/usr/local/maven/boot/plexus-classworlds-2.5.2.jar:/usr/local/maven/conf/logging jenkins.maven3.agent.Maven31Main /usr/local/maven /data/jenkins/slave.jar /data/jenkins/maven31-interceptor.jar /data/jenkins/maven3-interceptor-commons.jar 45631
Exception in thread "main" java.lang.UnsupportedClassVersionError: hudson/remoting/Launcher : Unsupported major.minor version 52.0
        at java.lang.ClassLoader.defineClass1(Native Method)
        at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
        at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
        at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
        at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
        at org.codehaus.plexus.classworlds.realm.ClassRealm.loadClassFromSelf(ClassRealm.java:401)
        at org.codehaus.plexus.classworlds.strategy.SelfFirstStrategy.loadClass(SelfFirstStrategy.java:42)
        at org.codehaus.plexus.classworlds.realm.ClassRealm.unsynchronizedLoadClass(ClassRealm.java:271)
        at org.codehaus.plexus.classworlds.realm.ClassRealm.loadClass(ClassRealm.java:247)
        at org.codehaus.plexus.classworlds.realm.ClassRealm.loadClass(ClassRealm.java:239)
        at jenkins.maven3.agent.Maven31Main.main(Maven31Main.java:138)
        at jenkins.maven3.agent.Maven31Main.main(Maven31Main.java:67)

经定位,原来 Maven 类型 Job 与 JDK 版本有一定兼容性,部分说明如下:

Maven jobs and Java versions compatibility : Because java serialized classes are exchanged between Jenkins master and Maven Jobs it is required that the JVM used to launch Maven is superior or equal to the version of Java for which Jenkins Master is built for.

  • Jenkins >= 1.520 requires Java 6 thus Maven jobs must be launched with Java >= 6.
  • Jenkins >= 1.612 requires Java 7 thus Maven jobs must be launched with Java >= 7.
  • Jenkins >= 2.54 requires Java 8 thus Maven jobs must be launched with Java >= 8.

详见:https://wiki.jenkins.io/display/JENKINS/Maven+Project+Plugin

而我们的这些 Job 必须使用指定 JDK 版本构建,不能随意修改 JDK 版本
而自由风格类型的 Job,不会出现这个问题
所以,计划将 Maven 类型 Job 迁移到自由风格类型

如何迁移?

那么如何迁移呢?
统计了下 Maven 类型的 Job,有数百个,手动迁移不现实的
通过 google 发现有前人写的 grovvy 脚本:https://github.com/akomakom/jenkins-scripts/blob/master/maven-to-freestyle.groovy
尝试运行该脚本,可能与环境有关,它会报错
由于对 groovy 不太熟悉,解决起来比较麻烦
所以参考它的逻辑,使用 python 脚本实现了迁移的功能

python 脚本如下所示,有需要的可以自行修改:

# -*- coding:utf-8 -*-
import xml.etree.ElementTree as ET

import sys
from jenkinsapi.jenkins import Jenkins

reload(sys)
sys.setdefaultencoding('utf8')


def get_jenkins_server():
    JENKINS_URL = "http://127.0.0.1:8080/"
    JENKINS_USERNAME = "jenkins"
    JENKINS_PASSWORD = "jenkins"
    return Jenkins(JENKINS_URL, username=JENKINS_USERNAME, password=JENKINS_PASSWORD, timeout=30)


def get_modified_xml(job_config_xml_tree):
    # create builders node
    builders_element = ET.Element("builders")
    job_config_xml_tree.append(builders_element)

    # Move prebuilders to builders
    prebuilders_node = job_config_xml_tree.find("prebuilders")
    if prebuilders_node is not None and prebuilders_node.getchildren() is not None:
        for pre_builder in prebuilders_node.getchildren():
            builders_element.append(pre_builder)
    job_config_xml_tree.remove(prebuilders_node)

    # Create a maven block
    maven_node = ET.Element("hudson.tasks.Maven")
    builders_element.append(maven_node)
    move(job_config_xml_tree.find("goals"), job_config_xml_tree, maven_node, "targets")
    maven_name_node = job_config_xml_tree.find("mavenName")
    if maven_name_node:
        move(maven_name_node, job_config_xml_tree, maven_node)
    else:
        maven_name_node = ET.Element("mavenName")
        maven_name_node.text = "Maven"
        maven_node.append(maven_name_node)
    move(job_config_xml_tree.find("rootPOM"), job_config_xml_tree, maven_node)
    move(job_config_xml_tree.find("mavenOpts"), job_config_xml_tree, maven_node)
    move(job_config_xml_tree.find("settings"), job_config_xml_tree, maven_node)
    move(job_config_xml_tree.find("globalSettings"), job_config_xml_tree, maven_node)

    # items that don't exist in maven step:
    remove(job_config_xml_tree, [
        'resolveDependencies',
        'processPlugins',
        'siteArchivingDisabled',
        'archivingDisabled',
        'mavenValidationLevel',
        'disableTriggerDownstreamProjects',
        'blockTriggerWhenBuilding',
        'fingerprintingDisabled',
        'incrementalBuild',
        'processPlugins',
        'siteArchivingDisabled',
        'ignoreUpstremChanges',
        'rootModule'
    ])

    # Move postbuilders to builders
    postbuilders_node = job_config_xml_tree.find("postbuilders")
    if postbuilders_node is not None and postbuilders_node.getchildren() is not None:
        for post_builder in postbuilders_node.getchildren():
            builders_element.append(post_builder)
    job_config_xml_tree.remove(postbuilders_node)

    # rename top-level element
    # new parent node
    new_job_config_xml_tree = ET.Element("project")
    for child_node in job_config_xml_tree.getchildren():
        new_job_config_xml_tree.append(child_node)

    return new_job_config_xml_tree


def move(from_node, from_parent_node, to_node, to_name=None):
    if from_node is not None:
        if to_name:
            tmp_element_node = ET.Element(to_name)
            tmp_element_node.text = from_node.text
            tmp_element_node.attrib = from_node.attrib
            to_node.append(tmp_element_node)
            from_parent_node.remove(from_node)
        else:
            to_node.append(from_node)
            from_parent_node.remove(from_node)


def remove(from_node, name_list):
    for name in name_list:
        tmp_node = from_node.find(name)
        if tmp_node is not None:
            from_node.remove(tmp_node)


def main():
    jenkins_server = get_jenkins_server()

    job_list = jenkins_server.get_jobs_list()
    for job_name in job_list:
        if not job_name.endswith("-DEPRECATED"):
            job_obj = jenkins_server.get_job(job_name)
            config_text = job_obj.get_config()
            config_xml_tree = ET.fromstring(config_text)
            if config_xml_tree.tag == "maven2-moduleset":
                new_config_xml_tree = get_modified_xml(config_xml_tree)
                new_config_xml = ET.tostring(new_config_xml_tree)
                jenkins_server.rename_job(job_name, job_name + "-DEPRECATED")
                jenkins_server.create_job(job_name, new_config_xml)
                print("migrate job from Maven Type to FreeStyle Type: " + job_name)


if __name__ == "__main__":
    main()

参考

2赞

我的迁移方案是,把之前旧版本的 jenkins 工程目录除了 workspace 目录不打包,其他全部打包,然后直接安装 2.X 版本,启动服务运行完成后,停止新安装版本的 jenkins 服务。在把我之前打包压缩的 jenkins工程解压,在新安装指定 jenkins 工作目录到解压的旧工程下就可以了,之前的所有配置全部都还在。
不知道有人像我这样迁移过吗 :slightly_smiling_face:

1赞

下面是我升级的一些说明和步骤
切记一定要备份 JENKINS_HOME 重要文件
升级前要做演练

环境说明:

  • 操作系统:CentOS release 6.8 (Final)
  • 目前 Jenkins 版本:1.638
  • 要升级到的 Jenkins 版本:2.190.2
  • 默认 JENKINS_HOME:/var/lib/jenkins/
  • 配置文件:/etc/sysconfig/jenkins
  • 日志目录:/var/log/jenkins/
  • war 包路径:/usr/lib/jenkins/jenkins.war
  • war 缓存路径:/var/cache/jenkins/

升级步骤:

  • 安装 JDK 8(Jenkins 新版本需要 JDK 版本为:[8, 11],详见https://jenkins.io/doc/administration/requirements/java/)
  • 替换 war 包(从清华大学镜像源下载)
  • 重启 Jenkins
    • 停止Jenkins
    • 清理缓存(rm -rf /var/cache/jenkins/war)
    • 启动 Jenkins
  • 节点配置:启动的 Java 版本修改为 8
  • 修复插件兼容性问题,安装或更新其依赖插件

参与今晚九点线上分享方式,关注[jenkins]公众号,回复“社区活动”即可进群获取线上分享地址。