summaryrefslogtreecommitdiffstats
path: root/utils
AgeCommit message (Collapse)AuthorFilesLines
2020-07-16SPEC-3414: add new metadata for kernelCICorentin LABBE1-0/+5
Recent kernelCI expects to have more metadata in LAVA job definition. This patchs add them. Change-Id: I936947ee2ea799c28220f25a587392b7d94d488e Bug-AGL: SPEC-3414 Signed-off-by: Corentin LABBE <clabbe@baylibre.com>
2020-06-11SPEC-3408: convert releng-scripts to python3Corentin LABBE3-16/+17
This patch convert releng-scripts to python3. While at it, let's creating requirements.txt for specifying depencies yaml and jinja2. Change-Id: Iba7b45b10fd7e1336bf1e4a73e0e6275392e373c Bug-AGL: SPEC-3408 Signed-off-by: Corentin LABBE <clabbe@baylibre.com>
2020-06-02Add prerelease type to generatorjellyfish_9.99.1jellyfish/9.99.19.99.1Jan-Simon Moeller3-5/+10
Allow the tool to generate jobs against the prerelease location. Signed-off-by: Jan-Simon Moeller <jsmoeller@linuxfoundation.org> Change-Id: I7f00ee52ad85b2aecc52679da4e98857a5dd59cb
2020-05-12Support the rewitten rpi4 machine file layoutJan-Simon Moeller1-0/+6
The reworked rpi4 board support will build with MACHINE = raspberrypi4-64 to match upstream. This leads to some paths being different on the download server. Thus we need to adapt the scripts. Signed-off-by: Jan-Simon Moeller <jsmoeller@linuxfoundation.org> Change-Id: Iac1e2bb8c80bb751138969653a5548c6133a665a
2020-05-04SPEC-3210: use the new rootfs name for qemuCorentin LABBE1-1/+1
When changing the name of the rootfs of qemu from ramdisk to rootvd, utils/job-prereq.py was not handled. This patch fix it. Bug-AGL: SPEC-3210 Change-Id: I17c5c52d4b5f605fc043deb96694f7935b5dc9f8 Signed-off-by: Corentin LABBE <clabbe@baylibre.com>
2020-04-07Add initial machine_branch declarationCorentin LABBE1-0/+1
CI fail with: UnboundLocalError: local variable 'machine_branch' referenced before assignment (See https://build.automotivelinux.org/job/ci-platform-meta-agl-verify-CIBT-qemuarm/568/console) This patch add an initial declaration for it Change-Id: I7fa7b836e1546f65cf6678c8527698208fc5581a Signed-off-by: Corentin LABBE <clabbe@baylibre.com>
2020-04-06Fix case when vcs_branch is not setJan-Simon Moeller1-3/+2
Signed-off-by: Jan-Simon Moeller <jsmoeller@linuxfoundation.org> Change-Id: I9d6f5cb51501b1cdfd9a643c28af96c48323704f
2020-01-24SPEC-3132: upsquare does not have dtbicefish_8.99.5icefish/8.99.58.99.5Corentin LABBE1-1/+1
Asking dtb for upsquare lead to utils/job-prereq.py crash since x86 machines does not have dtb. This patch made upsquare to use FILE_MAP_X86 Bug-AGL: SPEC-3132 Change-Id: I4d33b61719c0d5e3afd83e5d8bb8dc468e26e259 Signed-off-by: Corentin LABBE <clabbe@baylibre.com>
2019-11-20SPEC-2921: Use the right build_name for h3ulcb-kfCorentin LABBE1-0/+4
The build name for h3ulcb-kf is h3ulcb-nogfx. This patch adds the same hack than for m3ulcb. Bug-AGL: SPEC-2921 Change-Id: Ib687449e911ec0533cac811b648dd02cc8ae286f Signed-off-by: Corentin LABBE <clabbe@baylibre.com>
2019-08-08Add/use the new arguemnt --build-tagsKhouloud Touil2-2/+5
Add new argument --build-tags to be used in the job template and to be passed to the tests to be run along with using --device-tags. Bug-AGL: SPEC-2721 Signed-off-by: Khouloud Touil <ktouil@baylibre.com> Change-Id: I2fb4c3d0cb8cd1d0e3c36c1df55185cb4dde43c3
2019-06-19utils/job-prereq.py: Use yaml safe_loadhalibut_7.99.3halibut_7.99.2halibut/7.99.3halibut/7.99.27.99.37.99.2Corentin LABBE1-1/+1
On my gentoo, using yaml.load now give: Traceback (most recent call last): File "./utils/job-prereq.py", line 81, in <module> main() File "./utils/job-prereq.py", line 66, in main job_yaml = yaml.load(job) File "/usr/lib64/python2.7/site-packages/yaml/__init__.py", line 109, in load raise RuntimeError("Unsafe load() call disabled by Gentoo. See bug #659348") RuntimeError: Unsafe load() call disabled by Gentoo. See bug #659348 Note that on recent ubuntu, a warning appears also. (like on AGL jenkins) /w/workspace/ci-platform-meta-agl-demo-verify-CIBT-qemuarm64/repoclone/releng-scripts/utils/job-prereq.py:66: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details. job_yaml = yaml.load(job) This is due to a security risk of using yaml.load() Since releng-script does not rely on any behavour provided by load(), let's convert the call to safe_load(). This will also clean/reduce the output of jenkins logs Change-Id: I6158dcc21c0f7ec4645fb7b16acfaf1a6963cb26 Signed-off-by: Corentin LABBE <clabbe@baylibre.com>
2019-06-19Fix create-jobs for snapshot/daily/weekly build-typeKhouloud Touil1-0/+4
This patch take care of the snapshot/daily/weekly build-types if they are requested without mentioning the branch and version args. Bug-AGL: SPEC-2471 Change-Id: I86e5465162cc20be82b33feedcbed7f78300053d Signed-off-by: Khouloud Touil <ktouil@baylibre.com>
2019-06-06agljobtemplate: fix url generation for m3ulcbCorentin LABBE1-2/+4
When adding support for upsquare the URL generation for m3ulcb was broken. This patchs fix the generation by adding a new variable for the build name. Fixes: 32b88dfd726b ("SPEC-2075: fix URL generation for upsquare non-CI build") Bug-AGL: SPEC-2470 Signed-off-by: Corentin LABBE <clabbe@baylibre.com> Change-Id: I84bc2d8bbcbc49b048bbd95f84a531bde9ac6aba
2019-06-06Add support for qemuarm/qemuarm64 in utils/job-prereq.pyCorentin LABBE1-1/+1
This patchs permits utils/job-prereq.py to work with qemuarm/qemuarm64 Bug-AGL: SPEC-2376 Signed-off-by: Corentin LABBE <clabbe@baylibre.com> Change-Id: Ie8615a9ab4d1f0053afc098db508814e6aad1bf7
2019-05-30add tags: section, ensure tags are always lowercaseKevin Hilman1-1/+0
Whenever --device-tags are passed, a "tags:" section is added to the job with all of the tags passed on the command-line. To facilitate this, the device_tags list is passed directly from python to the jinja templates, and iteration is done in jinja instead of a string created in python. Also, while here, ensure that tags are always passed lower-cased. IMPORTANT: this implies that any tags in LAVA labs that are upper-case will *never* be used. Change-Id: I8f274098a69e34c32c5ed2d1c430eca0d364bcc5 Signed-off-by: Kevin Hilman <khilman@baylibre.com>
2019-05-22SPEC-2075: fix URL generation for upsquare non-CI buildCorentin LABBE1-1/+1
When adding support for upsquare, I have added a new machine_frag_url which contains the machine to use in url. It works for CI build type, but for other I miss to convert one machine to machine_frag_url Bug-AGL: SPEC-2075 Change-Id: Ieaaf71b62425cf73fb9582b84666640c4e9d6cae Signed-off-by: Corentin LABBE <clabbe@baylibre.com>
2019-05-15SPEC-1850: permit to override master branch for appfwCorentin LABBE2-2/+5
I have assumed that only master is used for appfw but it is false. Some app review are done against non-master branch. This patch adds a way to change the branch used via --app-branch. Bug-AGL: SPEC-1850 Change-Id: I10c8ad0e5a08158f9c2295692e4b3c266b97b14e Signed-off-by: Corentin LABBE <clabbe@baylibre.com>
2019-05-03utils/agljobtemplate.py: Fix APPURL generationCorentin LABBE1-8/+9
The generated APPURL was invalid: - it needed to be always using ci - it missed branch and arches path This patch move the generation of the application url in the jinja file for accessing the new sdk_arch Bug-AGL: SPEC-1850 Change-Id: Ie3ffff5f6181fdc89c3ebbd41078e4d60a06992b Signed-off-by: Corentin LABBE <clabbe@baylibre.com>
2019-05-03Remove porterCorentin LABBE1-1/+1
As asked by JSM on review 21232, this patch removes the porter machine. Bug-AGL: SPEC-2375 Change-Id: Iaa8083a271bb61ee1f9208f671d617fc15a00028 Signed-off-by: Corentin LABBE <clabbe@baylibre.com>
2019-04-18Permit to generate jobs for the upsquareCorentin LABBE1-3/+5
This patch adds suppport for the upsquare. Upsquare use a GRUB boot method, so we needed to add a new generic-grub-tftp boot template. Since all x86_64 use the same yocto generated build name, we add a new upsquare machine, but tweaked a bit utils/agljobtemplate.py to keep intel-corei7-64 everywhere (except for the LAVA device-type). Bug-AGL: SPEC-2075 Change-Id: I15c850fada53efbcda4fd58527d7b7414865f89d Signed-off-by: Corentin LABBE <clabbe@baylibre.com>
2019-04-16Modify the releng-scripts to deal with branches and dtb namesKhouloud Touil2-5/+14
This patch takes the branch and machine arguments and decide which is the dtb name to be used. Change-Id: I09bd67de51920c81780a77be396906a15d8247f5 Signed-off-by: Khouloud Touil <ktouil@baylibre.com>
2019-04-11Implement SPEC-1850: Add application-lifecycleCorentin LABBE2-2/+25
This patch adds a new parameter application_url which permit to adds a location where to download apps and test them. The applications_url could also be generated via app-changeid/app-patchset. Change-Id: I88d91b5e6e30b5ab5d72e8181b15b0f561bea72d Signed-off-by: Corentin LABBE <clabbe@baylibre.com>
2019-04-11Fix the use of the --build-type argumentKhouloud Touil1-1/+1
This patch fixes the use of the --build-type to take only the first part of it, which is the build type. Change-Id: Ic2f631a639babbd075bb005b2d33415e4ca916ab Signed-off-by: Khouloud Touil <ktouil@baylibre.com>
2019-03-22Fix the job parametersKhouloud Touil1-2/+8
This patch fixes the job parameters, if the build type is ci the parameters will be parsed as changeid and patchset otherwise it will be parsed as branch and version. Change-Id: I3f9abfd91a1b551974dd82ea82dc97a2e0a11d95 Signed-off-by: Khouloud Touil <ktouil@baylibre.com>
2019-03-15create-jobs: fix the error checkKhouloud Touil1-2/+2
This patch fixes the error check to permit the job creation when inserting only the --url arguement and not the branch/version or the changeid/patchsetm, to grant the job creation with a custom url. Change-Id: I625e13cacee8da6df569ab90038318c0994ec1c6 Signed-off-by: Khouloud Touil <ktouil@baylibre.com>
2019-03-07job-prereq: fix use of render_job after updatesKevin Hilman1-2/+2
Change-Id: I094220e829686f9acd362339fea93f947f5bfa57 Signed-off-by: Kevin Hilman <khilman@baylibre.com>
2019-02-28Modify the url_fragmentKhouloud Touil1-1/+3
Following all the modification and to keep things correct, this patch modify the url_fragment to gave a correct url following the build type if it is 'ci' or 'release' or other. Change-Id: I7122d5c4c31e37337ca68af3d86313d36e5b8f5b Signed-off-by: Khouloud Touil <ktouil@baylibre.com>
2019-02-28Add new metadata git.urlKhouloud Touil1-0/+6
This patch adds a new metadata field git.url, this is a link back to the AGL gerrit. Change-Id: I7f7b5fff79198fac7814004ab99b3687cf92125c Signed-off-by: Khouloud Touil <ktouil@baylibre.com>
2019-02-28Add new --build-id argumentKhouloud Touil2-2/+7
This patch adds a new build-id arguement that will fill the 'kernel_defconfig' in the metadata field with the build id extracted from the build-info file. Change-Id: I77bcc253a41d67d97afd7db32b8c8560d4309145 Signed-off-by: Khouloud Touil <ktouil@baylibre.com>
2019-02-28Reuse the build_version to fill the kernel_versionKhouloud Touil1-17/+4
The build_version will fill the kernel_version with the right value, this value will be the 'DISTRO_VERSION' extracted from the build-info file. Change-Id: Iecfdfd587dc15d53ea872d45b1916d33f6e382bf Signed-off-by: Khouloud Touil <ktouil@baylibre.com>
2019-02-28Use the --branch argument differentlyKhouloud Touil2-11/+5
Change the use of the --branch argument to display in the vsc_branch metadata field the AGL branch used in the build and get ride of the hard coded value "agl-branch". Change-Id: I9ff371248569c40ab9980347da14110e5227e1db Signed-off-by: Khouloud Touil <ktouil@baylibre.com>
2019-02-28Add new vcs_commit argumentKhouloud Touil2-2/+7
Add new vcs_commit arguement that will display in the metadeta field the build hash information. Change-Id: I079dfb8efda2ce82677e867804b3937d5a1f13c2 Signed-off-by: Khouloud Touil <ktouil@baylibre.com>
2019-02-28Separate --branch/--changeid --version/--patchsetKhouloud Touil2-11/+19
This patch separates the --branch/--changeid and --version/--patchset arguments to use each one of them differently. As well it renames url_branch to changeid and url_version patchset. It modifies also the error check to follow the other changes and be sure that the needed arguments are well inserted. Change-Id: If64ae3620b1519fc391e79d954247938cffcb491 Signed-off-by: Khouloud Touil <ktouil@baylibre.com>
2019-01-24Add new device-tags featureKhouloud Touil2-3/+7
utils/create-jobs: Add the --device-tags argument Add new --device-tags arguement to create a job that contain the device tags as parameters. utils/agljobtemplate: Add the device_tags to the job Add the device_tags to the job in order to be used later in the test scripts to determine which test case should be runned or not. Change-Id: If22fb40b2ddf1a7c0496a59d4318e633028495b9 Signed-off-by: Khouloud Touil <ktouil@baylibre.com>
2018-12-05templates: callbacks: support multiple callback targetsLoys Ollivier2-5/+7
LAVA jobs can now submit multiple callbacks per job. Update the create-jobs tool and its underlying templates to support this feature. - Callback templates are now split in two file and follow LAVA specification: 'section' and 'subsection'. - create-jobs tool supports multiple arguments for the '--callback-to' option. - Default configuration for callback targets can now be a list of targets. Change-Id: I36c502c0bb5b54010ce32a53ba78a6be5e6a1642 Signed-off-by: Loys Ollivier <lollivier@baylibre.com>
2018-11-21templates: add job generation support for iPXE based boardsguppy_6.99.1guppy/6.99.16.99.1Liu Wenlong1-6/+14
Since iPXE based boards can use NBD boot[1], now in releng-scripts, we also add support to generate LAVA jobs for iPXE based boards. With this support, Intel-UP2, one of the community reference boards, can do validation tests with the generated jobs easily. [1] https://git.lavasoftware.org/lava/lava/merge_requests/64/diffs Change-Id: I9adce67d7e9207181675e82bf22a653c73682eb8 Signed-off-by: Liu Wenlong <liuwl.fnst@cn.fujitsu.com>
2018-10-15agljobtemplate: fix UnboundLocalError for build_verson_defaultKevin Hilman1-0/+1
Change-Id: If8c422f8da34d6ab3474200095aff1f8cb7923d0 Signed-off-by: Kevin Hilman <khilman@baylibre.com>
2018-10-06agljobtemplate: CI builds: rename branch to gerritKevin Hilman1-1/+1
Change-Id: I8341bb7c3ab65a02a7b144171f778c850484850b Signed-off-by: Kevin Hilman <khilman@baylibre.com>
2018-10-06agljobtemplate: allow cmdline build-version to overrideKevin Hilman1-4/+5
Allow the --build-version cmdline option to override the other ways of generating the kernel.version metadata. Change-Id: I9da1de7fdc795d888a7b864965755821a53e2830 Signed-off-by: Kevin Hilman <khilman@baylibre.com>
2018-09-28Extend timeouts, add callback and gitignoreJan-Simon Möller1-1/+1
Change-Id: Ib28d25e1efd0033458fa9ae787610a8de9f5d275 Signed-off-by: Jan-Simon Möller <jsmoeller@linuxfoundation.org>
2018-09-27Add shortcut script for ci testsJan-Simon Möller1-0/+19
call with ./run-flounder-ci-tests 5.99.x lava-tool needs the auth-token setup. Signed-off-by: Jan-Simon Möller <jsmoeller@linuxfoundation.org>
2018-09-12fix metadata for ci build-typeguppy_6.90.0guppy/6.90.06.90.0Kevin Hilman1-2/+14
When using --build-type=ci, the url_branch and url_version are repurposed for gerrit changeid and patchset. However, this means that the kernelCI metadata fields "branch" and "version" values are interpreted as integers by the backend, causing various problems. To add clarity to kernelCI fields, prefix with "changeid-" and "patchset-", and this also makes kernelci-backend interpret fields as strings, as expected. Change-Id: I6b648d9c94751a3ce20ec48c593d60b7e71640b1 Signed-off-by: Kevin Hilman <khilman@baylibre.com>
2018-06-26utils/agljobtemplate: fixup image_type, branch and versionKevin Hilman1-0/+7
Fixup LAVA job metadata that is used by kernelCI UI for displaying branch, and version info, and also set "image-type" to the AGL build type (e.g., gerrit, snapshot, release) Change-Id: I12ca93dfbe8629bde5dc35ab1f056f8331805e40 Signed-off-by: Kevin Hilman <khilman@baylibre.com>
2018-01-18job-prereq: Fix the tool after build changesLoys Ollivier2-2/+2
The job-prereq tool got broken by some latest changes on the build/url scheme. Make the url argument for agljobtemplate optionnal. Pass build_type argument as build_type now that it is supported. Fixes: 788b31a122c3 ("utils: change the url / build behavior scheme") Change-Id: Ie9a0fc728a5e31cabd32e6463f2c6f913e1bd2cf Signed-off-by: Loys Ollivier <lollivier@baylibre.com>
2018-01-16Merge changes I44ae2990,Ied4e9f80,I9ce801a7Jan-Simon Moeller2-34/+43
* changes: templates: fix the rootfs name for m3ulcb and rpi3 daily builds utils: define test-plans based on build-type utils: change the url / build behavior scheme
2018-01-10templates: fix the rootfs name for m3ulcb and rpi3 daily buildsLoys Ollivier1-0/+1
The rootfs filename has changed for the daily builds. Change the default to the new filename for both platforms. Change-Id: I44ae29902d882ab6f38dc14bba11cc8aee88f8c2 Signed-off-by: Loys Ollivier <lollivier@baylibre.com>
2018-01-10utils: define test-plans based on build-typeLoys Ollivier2-19/+26
As the list of test is expending, we want to be able to specify test plans grouping several test definitions. These test plans are build type specific. Create a config file that specifies the test plans for each build type. If the user does not specify the tests to run then use the default ones for the build-type. Change-Id: Ied4e9f80d3e42787174189cd08499a2906e500ef Signed-off-by: Loys Ollivier <lollivier@baylibre.com>
2018-01-10utils: change the url / build behavior schemeLoys Ollivier2-23/+24
This is the first patch of the serie to differentiate builds and urls. As of now we were using the --url option to specify a build-type. Add a new option --build-type which specifies which type of build it is. From this build-type if no url is specified, the url can be defaulted by a configuration file. This configuration file is specific per user, e.g. AGL, others... Change-Id: I9ce801a7518b78ee859c6c3bbcad3a89e884e832 Signed-off-by: Loys Ollivier <lollivier@baylibre.com>
2018-01-03utils: Verify that callback options are correct v2eel_5.0.0eel/5.0.05.0.0Loys Ollivier1-2/+2
'--callback-from' is mandatory but not '--callback-to'. Fix this and allow specifying only 'from' without 'to'. Fixes: 5c6cf2bf9c09 ("utils: Verify that callback options are correct") Change-Id: I0706950cee4ede00cdbe7a3d1ade5500cc568502 Signed-off-by: Loys Ollivier <lollivier@baylibre.com>
2017-12-27utils: Miscellaneous pep8 changesLoys Ollivier2-20/+24
Coding style changes to follow pep8 coding guidelines. Change-Id: I9a9e542fb1e24da699639e0c3fd8bf98ad51aa79 Signed-off-by: Loys Ollivier <lollivier@baylibre.com>