SPEC-3723: restructure meta-agl

Goal is to reach a minimal meta-agl-core as base for IVI and IC work at the same time.
Trim dependencies and move most 'demo' related recipes to meta-agl-demo.

v2: changed to bbapend + .inc , added description
v3: testbuild of all images
v4: restore -test packagegroup and -qa images, compare manifests and adapt packagegroups.
v5: rebased
v6: merged meta-agl-distro into meta-agl-core,
    due to dependency on meta-oe, moved -test packagegroup and -qa images
      to own layer meta-agl-core-test
v7: Fixed comments from Paul Barker
v8: Update the markdown files
v9: restore wayland/weston/agl-compositor recipes/appends, reworked to
    move app f/w specific changes to bbappends in meta-app-framework and
    only demo specific weston-init changes to meta-agl-demo
v10: fix s/agldemo/aglcore/ missed in weston-init.bbappend

Description:

This patch is part 1 out of 2 large patches that implement the layer rework
discussed during the previous workshop. Essentially meta-agl-core is the
small but versatile new core layer of AGL serving as basis for
the work done by the IC and IVI EGs.
All demo related work is moved to meta-agl-demo in the 2nd patchset.
This should be applied together as atomic change.

The resulting meta-agl/* follows these guidelines:
- only bsp adaptations in meta-agl-bsp
- remove the agl-profile-* layers for simplicity
-- the packagegroup-agl(-profile)-graphical and so on
   have been kept in meta-agl-demo
- meta-agl-profile-core is now meta-agl-core
- meta-agl-core does pass yocto-check-layer
-- therefore use the bbappend + conditional + .inc file
   construct found in meta-virtualization
- meta-agl/meta-security has been merged into meta-agl/meta-app-framework
- meta-netboot does pass yocto-check-layer
- meta-pipewire does pass yocto-check-layer

Migration:
All packagegroups are preserved but they're now enabled by 'agl-demo'.

Bug-AGL: SPEC-3723

Signed-off-by: Jan-Simon Moeller <jsmoeller@linuxfoundation.org>
Signed-off-by: Scott Murray <scott.murray@konsulko.com>
Change-Id: Ia6c6e5e6ce2b4ffa69ea94959cdc57c310ba7c53
Reviewed-on: https://gerrit.automotivelinux.org/gerrit/c/AGL/meta-agl/+/25769

6 files changed, 309 insertions, 0 deletions

diff --git a/meta-agl-core/conf/distro/include/aarch64-tune.inc b/meta-agl-core/conf/distro/include/aarch64-tune.inc
new file mode 100644
index 000000000..553bb5a64
--- /dev/null
+++ b/meta-agl-core/conf/distro/include/aarch64-tune.inc
@@ -0,0 +1,13 @@
+# DEFAULTTUNE overrides for AGL on aarch64
+
+# We should not need the tuning include below as the BSP should include the right set already.
+# A double inclusion would produce a warning. This include line is just for reference
+# include conf/machine/include/arm/arch-armv8.inc
+
+DEFAULTTUNE_poky-agl := "aarch64"
+# not yet ready
+#QB_SYSTEM_NAME ?= "qemu-system-aarch"
+#QB_DEFAULT_KERNEL ?= "zImage"
+#QB_DEFAULT_FSTYPE ?= "ext4"
+#QB_MACHINE ?= "-machine foo"
+#QB_CPU ?= "-cpu bar"
diff --git a/meta-agl-core/conf/distro/include/arc-tune.inc b/meta-agl-core/conf/distro/include/arc-tune.inc
new file mode 100644
index 000000000..43dabd5a9
--- /dev/null
+++ b/meta-agl-core/conf/distro/include/arc-tune.inc
@@ -0,0 +1,3 @@
+# DEFAULTTUNE overrides for AGL on arc
+
+# DEFAULTTUNE ?= "archs"
diff --git a/meta-agl-core/conf/distro/include/arm-tune.inc b/meta-agl-core/conf/distro/include/arm-tune.inc
new file mode 100644
index 000000000..fa68e05a7
--- /dev/null
+++ b/meta-agl-core/conf/distro/include/arm-tune.inc
@@ -0,0 +1,23 @@
+# DEFAULTTUNE overrides for AGL on ARM 32bit
+
+# We should not need the tuning include below as the BSP should include the right set already.
+# A double inclusion would produce a warning. This include line is just for reference
+#   include conf/machine/include/arm/arch-armv7ve.inc
+# medium profile boards need to
+#   include conf/machine/include/arch/arm-armv7-a.inc
+
+# Standard target for 32bit ARM (newer than cortex-a15)
+AGLDEFAULTTUNE = "armv7vethf-neon-vfpv4"
+
+# for cortex-a8, cortex-a9, cortex-a7 (=rpi 2/3)
+AGLDEFAULTTUNE := "${@bb.utils.contains('DISTRO_FEATURES', 'agl-medium-arm-compiler', 'armv7athf-neon', '${AGLDEFAULTTUNE}', d)}"
+
+# for armv6 (=rpi0/1)
+AGLDEFAULTTUNE := "${@bb.utils.contains('DISTRO_FEATURES', 'agl-low-arm-compiler', 'arm1176jzfshf', '${AGLDEFAULTTUNE}', d)}"
+DEFAULTTUNE := "${AGLDEFAULTTUNE}"
+
+QB_SYSTEM_NAME ?= "qemu-system-arm"
+QB_DEFAULT_KERNEL ?= "zImage"
+QB_DEFAULT_FSTYPE ?= "ext4"
+QB_MACHINE ?= "${@bb.utils.contains('DISTRO_FEATURES', 'agl-medium-arm-compiler', '-machine vexpress-a9', '-machine vexpress-a15', d)}"
+QB_CPU ?= "${@bb.utils.contains('DISTRO_FEATURES', 'agl-medium-arm-compiler', '-cpu cortex-a9', '-cpu cortex-a15', d)}"
diff --git a/meta-agl-core/conf/distro/include/riscv64-tune.inc b/meta-agl-core/conf/distro/include/riscv64-tune.inc
new file mode 100644
index 000000000..f8118dec3
--- /dev/null
+++ b/meta-agl-core/conf/distro/include/riscv64-tune.inc
@@ -0,0 +1,7 @@
+# DEFAULTTUNE overrides for AGL on riscv64
+
+# We should not need the tuning include below as the BSP should include the right set already.
+# A double inclusion would produce a warning. This include line is just for reference
+# include conf/machine/include/arm/arch-armv8.inc
+
+DEFAULTTUNE = "riscv64"
diff --git a/meta-agl-core/conf/distro/include/x86_64-tune.inc b/meta-agl-core/conf/distro/include/x86_64-tune.inc
new file mode 100644
index 000000000..9f2bdc867
--- /dev/null
+++ b/meta-agl-core/conf/distro/include/x86_64-tune.inc
@@ -0,0 +1,55 @@
+# DEFAULTTUNE overrides for AGL on x86-64
+
+# We should not need the tuning include below as the BSP should include the right set already.
+# A double inclusion would produce a warning. This include line is just for reference
+#include conf/machine/include/tune-corei7.inc
+DEFAULTTUNE = "corei7-64"
+
+# shortened copy of tune-corei7.inc due to bug in inclusion for tune-core2.inc
+# TUNE_ARCH is using .= x86-64 , if done twice, you get x86-64x86-64 as TUNE_ARCH :/
+# we should have at least the core2 tunings by default (=qemux86-64)
+
+# Extra tune features
+TUNEVALID[corei7] = "Enable corei7 specific processor optimizations"
+TUNE_CCARGS = "${@bb.utils.contains('TUNE_FEATURES', 'corei7', ' -march=corei7 -mtune=corei7 -mfpmath=sse -msse4.2', '', d)}"
+
+# Extra tune selections
+AVAILTUNES += "corei7-32"
+TUNE_FEATURES_tune-corei7-32 = "${TUNE_FEATURES_tune-x86} corei7"
+BASE_LIB_tune-corei7-32 = "lib"
+TUNE_PKGARCH_tune-corei7-32 = "corei7-32"
+PACKAGE_EXTRA_ARCHS_tune-corei7-32 = "${PACKAGE_EXTRA_ARCHS_tune-core2-32} corei7-32"
+QEMU_EXTRAOPTIONS_corei7-32 = " -cpu Nehalem,check=false"
+
+AVAILTUNES += "corei7-64"
+TUNE_FEATURES_tune-corei7-64 = "${TUNE_FEATURES_tune-x86-64} corei7"
+BASE_LIB_tune-corei7-64 = "lib64"
+TUNE_PKGARCH_tune-corei7-64 = "corei7-64"
+PACKAGE_EXTRA_ARCHS_tune-corei7-64 = "${PACKAGE_EXTRA_ARCHS_tune-core2-64} corei7-64"
+QEMU_EXTRAOPTIONS_corei7-64 = " -cpu Nehalem,check=false"
+
+AVAILTUNES += "corei7-64-x32"
+TUNE_FEATURES_tune-corei7-64-x32 = "${TUNE_FEATURES_tune-x86-64-x32} corei7"
+BASE_LIB_tune-corei7-64-x32 = "libx32"
+TUNE_PKGARCH_tune-corei7-64-x32 = "corei7-64-x32"
+PACKAGE_EXTRA_ARCHS_tune-corei7-64-x32 = "${PACKAGE_EXTRA_ARCHS_tune-core2-64-x32} corei7-64-x32"
+QEMU_EXTRAOPTIONS_corei7-64-x32 = " -cpu Nehalem,check=false"
+
+TUNE_ARCH = "${@bb.utils.contains('TUNE_FEATURES', 'm64', '${X86ARCH64}', '' ,d)}"
+
+# Bump qemux86-64 up to corei7
+AVAILTUNES += "qemux86-64"
+TUNE_FEATURES_tune-qemux86-64 = "${TUNE_FEATURES_tune-x86-64} corei7"
+BASE_LIB_tune-qemux86-64 = "lib64"
+TUNE_PKGARCH_tune-qemux86-64 = "corei7-64"
+PACKAGE_EXTRA_ARCHS_tune-qemux86-64 = "${PACKAGE_EXTRA_ARCHS_tune-core2-64} corei7-64"
+QEMU_EXTRAOPTIONS_qemux86-64 = " -cpu Nehalem,check=false"
+
+# QEMU / runqemu options (partly overrides qemuboot-x86.inc)
+QB_MACHINE = "-machine q35"
+QB_SYSTEM_NAME_x86-64 = "qemu-system-x86_64"
+QB_CPU_x86-64 = "-cpu qemu64,+ssse3,+sse4.1,+sse4.2,+popcnt"
+QB_CPU_KVM_x86-64 = "-cpu kvm64 -cpu qemu64,+ssse3,+sse4.1,+sse4.2,+popcnt"
+
+# Add the 'virtio-rng-pci' device otherwise the guest may run out of entropy
+QB_OPT_APPEND = "-vga vmware -show-cursor -usb -device usb-tablet -device virtio-rng-pci"
diff --git a/meta-agl-core/conf/distro/poky-agl.conf b/meta-agl-core/conf/distro/poky-agl.conf
new file mode 100644
index 000000000..63a1960af
--- /dev/null
+++ b/meta-agl-core/conf/distro/poky-agl.conf
@@ -0,0 +1,208 @@
+require conf/distro/poky.conf
+
+# AGL specific derivations
+DISTRO = "poky-agl"
+DISTRO_NAME = "Automotive Grade Linux"
+
+
+# Release flags
+DISTRO_CODENAME = "koi"
+AGL_BRANCH = "master"
+AGLVERSION = "10.90.0"
+# switch  devel/release
+AGLRELEASETYPE ?= "agldevelopment"
+#AGLRELEASETYPE ?= "aglrelease"
+OVERRIDES .= ":${AGLRELEASETYPE}"
+#
+
+#for development
+DISTRO_VERSION_agldevelopment := "${AGLVERSION}+snapshot-${DATE}"
+AGL_APP_REVISION_agldevelopment = "${AUTOREV}"
+AGL_DEFAULT_REVISION_agldevelopment = "${AUTOREV}"
+
+#for release
+DISTRO_VERSION_aglrelease := "${AGLVERSION}"
+DISTRO_BRANCH_VERSION_TAG = "${DISTRO_CODENAME}/${@'${DISTRO_VERSION}'.replace('+snapshot-${DATE}','')}"
+AGL_APP_REVISION_aglrelease := "refs/tags/${DISTRO_BRANCH_VERSION_TAG}"
+AGL_DEFAULT_REVISION_aglrelease := "refs/tags/${DISTRO_BRANCH_VERSION_TAG}"
+
+# reproducible builds:
+# enable the flag
+BUILD_REPRODUCIBLE_BINARIES = "1"
+# Set the desired timestamps
+export SOURCE_DATE_EPOCH = "1593400000"
+REPRODUCIBLE_TIMESTAMP_ROOTFS = "1593400000"
+# inherit the class
+INHERIT += "reproducible_build"
+
+
+# SDK
+SDK_VENDOR = "-aglsdk"
+SDK_VERSION := "${@'${DISTRO_VERSION}'.replace('snapshot-${DATE}','snapshot')}"
+
+# SDKPATH is the folder where the SDK is going to be installed
+# Due to an issue with the qt5 environment (see SPEC-1667),
+# we add DEFAULTTUNE to the SDKPATH to mitigate the issue.
+SDKPATH = "/opt/agl-sdk/${SDK_VERSION}-${DEFAULTTUNE}"
+
+
+MAINTAINER = "AGL https://lists.automotivelinux.org/g/agl-dev-community"
+
+TARGET_VENDOR = "-agl"
+
+# Override these in poky based distros
+AGL_DEFAULT_DISTRO_FEATURES = "largefile systemd opengl wayland pam bluetooth bluez5 3g polkit"
+POKY_DEFAULT_DISTRO_FEATURES := "${AGL_DEFAULT_DISTRO_FEATURES}"
+OVERRIDES .= ":${DISTRO_CODENAME}"
+
+QEMU_TARGETS ?= "arm aarch64 i386 x86_64 riscv64"
+# Other QEMU_TARGETS "mips mips64 mips64el ppc sh4"
+
+MIRRORS =+ "\
+bzr://.*/.*      https://download.automotivelinux.org/AGL/mirror/ \n \
+cvs://.*/.*      https://download.automotivelinux.org/AGL/mirror/ \n \
+git://.*/.*      https://download.automotivelinux.org/AGL/mirror/ \n \
+gitsm://.*/.*    https://download.automotivelinux.org/AGL/mirror/ \n \
+hg://.*/.*       https://download.automotivelinux.org/AGL/mirror/ \n \
+osc://.*/.*      https://download.automotivelinux.org/AGL/mirror/ \n \
+p4://.*/.*       https://download.automotivelinux.org/AGL/mirror/ \n \
+svn://.*/.*      https://download.automotivelinux.org/AGL/mirror/ \n \
+bzr://.*/.*      http://download.automotivelinux.org/AGL/mirror/  \n \
+cvs://.*/.*      http://download.automotivelinux.org/AGL/mirror/  \n \
+git://.*/.*      http://download.automotivelinux.org/AGL/mirror/  \n \
+gitsm://.*/.*    http://download.automotivelinux.org/AGL/mirror/  \n \
+hg://.*/.*       http://download.automotivelinux.org/AGL/mirror/  \n \
+osc://.*/.*      http://download.automotivelinux.org/AGL/mirror/  \n \
+p4://.*/.*       http://download.automotivelinux.org/AGL/mirror/  \n \
+svn://.*/.*      http://download.automotivelinux.org/AGL/mirror/  \n \
+                 \
+ftp://.*/.*      https://download.automotivelinux.org/AGL/mirror/ \n \
+http://.*/.*     https://download.automotivelinux.org/AGL/mirror/ \n \
+https://.*/.*    https://download.automotivelinux.org/AGL/mirror/ \n \
+ftp://.*/.*      http://download.automotivelinux.org/AGL/mirror/  \n \
+http://.*/.*     http://download.automotivelinux.org/AGL/mirror/  \n \
+https://.*/.*    http://download.automotivelinux.org/AGL/mirror/  \n \
+"
+
+# The CONNECTIVITY_CHECK_URI's are used to test whether we can succesfully
+# fetch from the network (and warn you if not). To disable the test set
+# the variable to be empty.
+# Git example url: git://git.yoctoproject.org/yocto-firewall-test;protocol=git;rev=HEAD
+CONNECTIVITY_CHECK_URIS ?= ""
+
+PREFERRED_PROVIDER_udev ?= "systemd"
+PREFERRED_PROVIDER_udev-utils ?= "systemd"
+
+VIRTUAL-RUNTIME_init_manager = "systemd"
+VIRTUAL-RUNTIME_initscripts = ""
+
+# network manager to use (possible values: systemd, connman)
+VIRTUAL-RUNTIME_net_manager = "connman" 
+
+DISTRO_FEATURES_APPEND = " "
+DISTRO_FEATURES_remove = "x11"
+DISTRO_FEATURES_BACKFILL_CONSIDERED = "sysvinit pulseaudio"
+
+# Prefer GStreamer 1.16.x by default
+PREFERRED_VERSION_gstreamer1.0              ?= "1.16.%"
+PREFERRED_VERSION_gstreamer1.0-plugins-bad  ?= "1.16.%"
+PREFERRED_VERSION_gstreamer1.0-plugins-base ?= "1.16.%"
+PREFERRED_VERSION_gstreamer1.0-plugins-good ?= "1.16.%"
+PREFERRED_VERSION_gstreamer1.0-plugins-ugly ?= "1.16.%"
+PREFERRED_VERSION_gstreamer1.0-libav        ?= "1.16.%"
+PREFERRED_VERSION_gstreamer1.0-omx          ?= "1.16.%"
+
+# Prefer libjpeg-turbo
+PREFERRED_PROVIDER_jpeg = "libjpeg-turbo"
+
+# using multiple BSP layers causes dangling bbappends in meta-agl-bsp
+# turn it into a warning
+#BB_DANGLINGAPPENDS_WARNONLY = "1"
+
+# Not yet upstreamed; should be submitted.
+SECURITY_CFLAGS_pn-qtwebengine = "${SECURITY_NO_PIE_CFLAGS}"
+
+#Redefined USER_CLASSES
+#Disable prelink. It is unmaintained and likely will go away.
+#default: USER_CLASSES ?= "buildstats image-mklibs image-prelink"
+USER_CLASSES = "buildstats image-mklibs"
+
+# AGL uses 4 optimization levels
+# 2 for ARM 32bit 
+#   - a high and a medium setting for the CCARGS
+#   - the high setting is default (needs >= cortex-a15)
+#   - the medium setting is enabled with: DISTRO_FEATURES_append = " agl-medium-arm-compiler "
+# 1 for ARM 64bit / AARCH64
+# 1 for x86-64
+# 1 for RISC-V 64-bit
+require conf/distro/include/${TARGET_ARCH}-tune.inc
+
+# Generic qemu and qemuboot (runqemu) enhancements
+# check qemuboot.bbclass
+# - use 2G RAM by default
+QB_MEM ?= "-m 2048"
+# use pulseaudio on the host side - off as qemu-native is built with alsa
+#QB_AUDIO_DRV = "pa"
+# expose a virtual 'hda' sound card to the guest (arm/aarch64/x86-64)
+QB_AUDIO_OPT = "-soundhw hda"
+
+# Board templates can add extra IMAGE_FSTYPES through this.
+# It is added (late) through the AGL image recipes.
+AGL_EXTRA_IMAGE_FSTYPES ??= ""
+AGL_EXTRA_INITRAMFS_FSTYPES ??= ""
+#
+# Default IMAGE FSTYPES wic.xz
+AGL_DEFAULT_IMAGE_FSTYPES ?= "wic.xz wic.bmap wic.xz.sha256sum"
+AGL_DEFAULT_IMAGE_FSTYPES_qemuall ?= "${@bb.utils.contains('DISTRO_FEATURES', 'AGLCI', 'ext4.xz', 'ext4', d)}"
+AGL_DEFAULT_IMAGE_FSTYPES_append_netboot = " ${@bb.utils.contains('DISTRO_FEATURES', 'AGLCI', 'ext4.xz', 'ext4', d)}"
+AGL_DEFAULT_INITRAMFS_FSTYPES ?= "ext4.gz"
+
+# DEFAULT IMAGE_FSTYPES for AGL (no - BSPs should not set this)
+#
+IMAGE_FSTYPES := "${AGL_DEFAULT_IMAGE_FSTYPES} ${AGL_EXTRA_IMAGE_FSTYPES}"
+INITRAMFS_FSTYPES := "${AGL_DEFAULT_INITRAMFS_FSTYPES} ${AGL_EXTRA_INITRAMFS_FSTYPES}"
+#
+# THE FUTURE is 'wic'
+# IMAGE_FSTYPES_append = " wic"
+
+# enable the packagemanagement
+IMAGE_FEATURES =+ "package-management"
+PACKAGE_FEED_URIS = "http://192.168.7.1:8000/tmp/deploy/"
+PACKAGE_FEED_BASE_PATHS = "rpm"
+PACKAGE_FEED_ARCHS = ""
+#${PACKAGE_ARCHS}"
+
+# Complementary package definitions for agl-test-wgt and agl-devel-wgt
+# image features
+COMPLEMENTARY_GLOB[agl-test-wgt] = '*-test'
+COMPLEMENTARY_GLOB[agl-devel-wgt] = '*-coverage *-debug'
+
+# Fix for rpm metadata clash between nativesdk-cmake and nativesdk-qtbase-tools
+# (revalidate after 2017-10-15, 2018-06-12: still required)
+DIRFILES_pn-nativesdk-cmake = "1"
+
+# For https://jira.automotivelinux.org/browse/SPEC-1629 :
+# We exclude the conflicting vars from the sstate task hash.
+# This needs to be verified.
+do_package_write_rpm[vardepsexclude] += "ARCHIVER_TOPDIR PF"
+
+# Set preferred provider to new navi application
+PREFERRED_RPROVIDER_virtual/navigation ?= "ondemandnavi"
+
+# Set preferred provider for low-can device mapping configuration
+PREFERRED_RPROVIDER_virtual/low-can-dev-mapping ?= "dev-mapping"
+
+# Mask upstream meta-security bbappend that interferes with non-linux-yocto
+# kernels due to the bug of having two wildcards, which bitbake does not
+# support. Not having it is not an issue, since there is also explicit
+# enabling of required security features done by meta-agl/meta-security.
+BBMASK += "meta-security/recipes-kernel/linux/linux-%_5.%.bbappend"
+
+# Workaround for https://jira.automotivelinux.org/browse/SPEC-3621
+# TLDR: do_image_wic will modify the target's /etc/fstab while
+# do_image_tar and do_image_ext4 can run in parallel
+# This needs to be fixed upstream:
+# See https://bugzilla.yoctoproject.org/show_bug.cgi?id=13994
+# For convenience restricted to ext4 and tar image types.
+AGL_FORBID_FSTAB_UPDATE ?= "${@bb.utils.contains_any('IMAGE_FSTYPES', 'ext4 ext4.xz tar tar.gz tar.bz2 tar.xz', '--no-fstab-update', '', d)}"
+WIC_CREATE_EXTRA_ARGS_append = " ${AGL_FORBID_FSTAB_UPDATE}"