diff options
Diffstat (limited to 'docs/part-2/1_xds-server')
| -rw-r--r-- | docs/part-2/1_xds-server/0_abstract.md | 41 | ||||
| -rw-r--r-- | docs/part-2/1_xds-server/1_config.md | 86 | ||||
| -rw-r--r-- | docs/part-2/1_xds-server/2_how-to-run.md | 238 | ||||
| -rw-r--r-- | docs/part-2/1_xds-server/3_build.md | 73 | ||||
| -rw-r--r-- | docs/part-2/1_xds-server/4_debug.md | 36 |
5 files changed, 474 insertions, 0 deletions
diff --git a/docs/part-2/1_xds-server/0_abstract.md b/docs/part-2/1_xds-server/0_abstract.md new file mode 100644 index 0000000..07dd057 --- /dev/null +++ b/docs/part-2/1_xds-server/0_abstract.md @@ -0,0 +1,41 @@ +# XDS - X(cross) Development System Server + +`xds-server` is a web server that allows user to remotely cross build applications. + +- The first goal is to provide a multi-platform cross development tool with near-zero installation. +- The second goal is to keep application sources locally (on user's machine). + - Make it compatible with existing IT policies (e.g. corporate backup or SCM). + - Let user to continue to work as usual. + - Use his favorite editor. + - keep performance while editing/browsing sources. + - Avoids manual operation + +This powerful and portable webserver (written in [Go](https://golang.org)) +exposes a REST interface over HTTP. + +`xds-server` uses [Syncthing](https://syncthing.net/) tool to synchronize +projects files from user machine to build server machine or container. + +`xds-server` is commonly running on a build server (within a container or not) +and [xds-agent](../2_xds-agent/0_abstract.html) must run on the developer/user machine in order +to setup the following connection chain: + +```schema + developer/user machine | build server or container + ---------------------------|----------------------------- + xds-cli <---> xds-agent <-|-> xds-server +``` + +**SEE ALSO**: [xds-cli](https://gerrit.automotivelinux.org/gerrit/gitweb?p=src/xds/xds-cli.git), +a command-line tool that allows you to send commands to `xds-agent / xds-server` +and for example build your application from command-line or from your favorite +IDE (such as Netbeans or Visual Studio Code) through `xds-agent <=> xds-server`. + +--- + +Links to subchapters : + +- [Configuration](./1_config.html) +- [How to run](./2_how-to-run.html) +- [Build from scratch](./3_build.html) +- [Debugging](./4_debug.html) diff --git a/docs/part-2/1_xds-server/1_config.md b/docs/part-2/1_xds-server/1_config.md new file mode 100644 index 0000000..594bdf2 --- /dev/null +++ b/docs/part-2/1_xds-server/1_config.md @@ -0,0 +1,86 @@ +# Configuration + +`xds-server` configuration is driven by a JSON config file (`server-config.json`). + +Here is the logic to determine which `server-config.json` file will be used: + +1. from command line option: `--config myConfig.json` +1. `$HOME/.xds/server/server-config.json` file +1. `/etc/xds/server/server-config.json` file +1. `<xds-server executable dir>/server-config.json` file + +Supported fields in configuration file are: + +- **httpPort** : HTTP port of client webapp/REST API +- **webAppDir** : location of client web application (default: webapp/dist) +- **shareRootDir** : root directory where projects will be copied +- **logsDir** : directory to store logs (eg. syncthing output) +- **sdkScriptsDir** : directory where scripts, used to managed SDKs, are installed +- **syncthing.binDir** : syncthing binaries directory (default: executable directory) +- **syncthing.home"** : syncthing home directory (usually .../syncthing-config) +- **syncthing.gui-address** : syncthing gui url (default <http://localhost:8385>) +- **syncthing.gui-apikey** : syncthing api-key to use (default auto-generated) + +All fields are optional and example below corresponds to the default values. + +```json +{ + "httpPort": 8000, + "webAppDir": "webapp/dist", + "shareRootDir": "${HOME}/.xds/server/projects", + "logsDir": "/tmp/logs", + "sdkScriptsDir": "${EXEPATH}/scripts/sdks", + "syncthing": { + "binDir": "./bin", + "home": "${HOME}/.xds/server/syncthing-config", + "gui-address": "http://localhost:8385", + "gui-apikey": "123456789", + } +} +``` + +>**Notes:** +> +>Environment variables are supported by using `${MY_VAR}` syntax. +> + +When `xds-server` is started as a systemd service, default environment variables +are set into `/etc/default/xds-server` file. + +`xds-server` configuration is also driven by a JSON config file (`server-config.json`), +and default JSON config is `/etc/xds/server/server-config.json`. + +<!-- note --> +**Note:** +You can use your own JSON config by settings `APP_CONFIG` variable of +`/etc/default/xds-server` file to your file, for example `/home/MYUSER/.xds/server/server-config.json` +<!-- endnote --> + +## Disable syncthing + +`CloudSync` synchronization type based on `syncthing` tool can be disabled by +simply removing (or renaming) `"syncthing"` key in configuration file. +Here is a JSON configuration file example where syncthing key as been renamed: + +```json +{ + "httpPort": 8000, + "webAppDir": "webapp/dist", + "shareRootDir": "${HOME}/.xds/server/projects", + "logsDir": "/tmp/logs", + "sdkScriptsDir": "${EXEPATH}/scripts/sdks", + "syncthing_DISABLE": { + "binDir": "./bin", + "home": "${HOME}/.xds/server/syncthing-config", + } +} +``` + +On benefit to do that is to increase XDS-Server startup time. + +<!-- note --> +**Note:** + +- `CloudSync` (AKA syncthing) synchronization type can also be disabled when `"syncthing"` key is not defined in JSON configuration file. + +<!-- endnote --> diff --git a/docs/part-2/1_xds-server/2_how-to-run.md b/docs/part-2/1_xds-server/2_how-to-run.md new file mode 100644 index 0000000..36bc9a6 --- /dev/null +++ b/docs/part-2/1_xds-server/2_how-to-run.md @@ -0,0 +1,238 @@ +# How to run + +`xds-server` has been designed to easily compile and debug +[AGL](https://www.automotivelinux.org/) applications. That's why `xds-server` has +been integrated into AGL SDK docker container. + +>**Note:** For more info about AGL SDK docker container, please refer to +[AGL SDK Quick Setup](http://docs.automotivelinux.org/docs/getting_started/en/dev/reference/setup-sdk-environment.html) + +## Start xds-server within the container + +See [Installation based on Docker container](../../part-1/2-1_install-xds-server-docker.html) +chapter. + +## Manually Start XDS server + +XDS server is started as a service by Systemd. + +```bash +/lib/systemd/system/xds-server.service +``` + +This Systemd service starts a bash script + +```bash +/opt/AGL/xds/server/xds-server-start.sh +``` + +If needed you can change default setting by defining specific environment +variables file + +```bash +ssh -t -p 2222 devel@localhost vim /etc/default/xds-server +``` + +For example to control log level, just set LOG_LEVEL env variable. + +knowing that supported *level* are: + +- panic +- fatal +- error +- warn +- info +- debug + +```bash +docker exec ${CONTAINER_NAME} bash -c "echo 'LOG_LEVEL=debug' >> /etc/default/xds-server" +ssh -p 2222 devel@localhost -- "systemctl --user restart xds-server" +``` + +## SDK cross-toolchain Management + +### Setup to add support of a new SDK family + +<!-- note --> +**Optional step**: read this chapter only if you plan to add a new SDK family. +<!-- endnote --> + +`xds-server` dynamically detects supported SDKs by scanning sub-directories of +`sdkScriptsDir` directory (see [Configuration chapter](1_config.html)). + +Each sub-directory (usually name is the same as the SDK family) of `sdkScriptsDir` +must contain a set of scripts that will be called by `xds-server` to managed SDKs +of a specific family. + +These scripts are: + +- `add`: used to add/install a new SDK +- `db-dump`: returned the list of available and installed SDKs (JSON format) +- `db-update`: update SDKs database +- `get-family-config`: returned SDK family configuration structure (JSON format) +- `get-sdk-info`: extract SDK info (JSON format) from a SDK file/tarball +- `remove`: used to remove an existing SDK + +For example, here 2 SDKs family (`agl` and `zephyr`) are defined: + +```bash +# > tree ./sdks/ +./sdks/ +├── agl +│ ├── add +│ ├── db-dump +│ ├── db-update +│ ├── get-family-config +│ ├── get-sdk-info +│ └── remove +├── README.md +└── zephyr + ├── add +│ ├── db-dump +│ ├── db-update +│ ├── get-family-config +│ ├── get-sdk-info + └── remove +``` + +On startup `xds-server` will call in order: + +- `sdks/*/get-family-config` to get configuration of each SDK family. +- `sdks/*/db-dump` scripts to get the initial list of available and installed SDKs. + +Please refer to `sdks/README.md` for more information about scripts definition +and to understand how to add support of a new SDK family. + +### Install a new SDK from command line + +Used `sdks` command of `xds-cli` tool to managed SDKs. + +```bash +# List all available SDKs +xds-cli sdks ls -a + +List of available SDKs: +ID NAME STATUS VERSION ARCH +ec15afe0 AGL-release-eel-4.99.4-raspberrypi3 Not Installed 4.99.4 armv7vehf-neon-vfpv4 +944d2d5a AGL-snapshots-master-latest-intel-corei7-64 Not Installed 4.99.3+snapshot corei7-64 +cf3a4365 AGL-release-dab-4.0.2-qemux86-64 Not Installed 4.0.2 corei7-64 +d65fe750 AGL-release-eel-latest-qemux86-64 Not Installed 4.99.3 corei7-64 +a0ae663d poky-agl-corei7-64-3.99.1+snapshot Installed 3.99.1+snapshot corei7-64 +87f0400b AGL-release-dab-3.99.3-m3ulcb-nogfx Installed 3.99.3 aarch64 +8c2f2841 AGL-release-dab-4.0.2-dragonboard-410c Not Installed 4.0.2 aarch64 +... + +# Install a SDK (downloaded + install) +xds-cli sdks install d65fe750 + +Installation of 'AGL-release-eel-latest-qemux86-64' SDK successfully started. +Downloading poky-agl-glibc-x86_64-agl-demo-platform-crosssdk-corei7-64-toolchain-4.99.5.sh ... +--2018-01-02 11:22:23-- https://download.automotivelinux.org/AGL/release/eel/latest/qemux86-64/deploy/sdk/poky-agl-glibc-x86_64-agl-demo-platform-crosssdk-corei7-64-toolchain-4.99.5.sh +Resolving download.automotivelinux.org (download.automotivelinux.org)... 199.19.213.77 +Connecting to download.automotivelinux.org (download.automotivelinux.org)|199.19.213.77|:443... connected. +HTTP request sent, awaiting response... 200 OK +Length: 665996704 (635M) [application/x-sh] +Saving to: ‘/tmp/tmp.wuQzLdImCS/poky-agl-glibc-x86_64-agl-demo-platform-crosssdk-corei7-64-toolchain-4.99.5.sh’ + + 0K .......... .......... .......... .......... .......... 0% 82,7K 2h11m + 50K .......... .......... .......... .......... .......... 0% 89,1K 2h6m + 100K .......... .......... .......... .......... .......... 0% 82,0K 2h8m +... + 50300K .......... .......... .......... .......... .......... 99% 2,15M 0s + 650350K .......... .......... .......... ....... 100% 4,04M=10m35s + +2018-01-02 12:17:06 (1024 KB/s) - ‘/tmp/tmp.CWyEj3z76Q/poky-agl-glibc-x86_64-agl-demo-platform-crosssdk-corei7-64-toolchain-4.99.5.sh’ saved [665996704/665996704] + +Automotive Grade Linux SDK installer version 5.0.0 +=================================================== +You are about to install the SDK to "/xdt/sdk/poky-agl/5.0.0/corei7-64". Proceed[Y/n]? Y +Extracting SDK....................................................................................................................done +Setting it up...done +SDK has been successfully set up and is ready to be used. +Each time you wish to use the SDK in a new shell session, you need to source the environment setup script e.g. + $ . /xdt/sdk/poky-agl/5.0.0/corei7-64/environment-setup-corei7-64-agl-linux + +SDK ID d65fe750-d3a7-38f5-83d8-3d3806054f8d successfully installed. + + +# Abort an installation that is in progress +xds-cli sdks abort -id d65fe750 + +# Install a SDK (using a local SDK package/file) +xds-cli sdks install --file $HOME/xds-workspace/sdks/poky-agl-glibc-x86_64-agl-demo-platform-crosssdk-corei7-64-toolchain-5.0.0.sh + +``` + +<!-- note --> +Installation based on a local SDK package is only supported when SDK file package +is located in `$HOME/xds-workspace/sdks` directory +<!-- endnote --> + +### Install a new SDK from XDS Dashboard + +Open XDS-Dashboard in web-browser and select `SDKs` entry in left side menu. + +Then switch to `SDKs MANAGEMENT` view : + +{:: style="margin:auto; display:flex"} + +Use filter boxes to find the SDK you want to install and then click on plus +icon (Actions column) to start installation. + +SDK download and installation process may take several minutes and output of installation script (output of `add` script mentioned in above chapter). + +{:: style="margin:auto; display:flex"} + +Note that you can abort installation by clicking on `CANCEL` button. + +### Un-install a SDK from command line + +Used `sdks` command of `xds-cli` tool to managed SDKs. + +```bash +# List installed SDKs +xds-cli sdks ls +List of installed SDKs: + ID NAME STATUS VERSION ARCH + c39e5998 poky-agl_aarch64_4.0.1 Installed 4.0.1 aarch64 + d610bfbf poky-agl-aarch64.current_on_iotbzh_download-3.99.1+snapshot Installed 3.99.1+snapshot aarch64.current_on_iotbzh_download + a0ae663d poky-agl-corei7-64-3.99.1+snapshot Installed 3.99.1+snapshot corei7-64 + 87f0400b AGL-release-dab-3.99.3-m3ulcb-nogfx Installed 3.99.3 aarch64 + 352c0584 poky-agl-corei7-64-3.99.2+snapshot Installed 3.99.2+snapshot corei7-64 + d65fe750 AGL-release-eel-latest-qemux86-64 Installed 4.99.5 corei7-64 + +# Un-install a SDK +xds-cli sdks uninstall d65fe750 +SDK ID d65fe750-d3a7-38f5-83d8-3d3806054f8d successfully deleted. +``` + +### Un-install a SDK from XDS Dashboard + +Open XDS-Dashboard in web-browser and select `SDKs` entry in left side menu. + +If needed, switch to `BASIC SDKS VIEW` view and click on trash icon located +in the top-right corner of SDK card. + +## XDS server REST API and Web application + +`xds-server` exposes a REST API and serves a basic web-application. + +REST API based url is `http://localhost:8000/api/v1/` when XDS server is +running on your host (localhost) and basic web-application is available at +[http://localhost:8000](http://localhost:8000). + +Just replace `localhost` by the host name or ip when `xds-server` is running +on another host. + +```bash +# Get version using REST API +curl http://localhost:8000/api/v1/version + +# Open browser and local xds-server web-application +xdg-open http://localhost:8000 +``` + +Then follow instructions provided on this page to install and start `xds-agent` +that must run locally on your machine. + +See also [xds-agent documentation](../2_xds-agent/0_abstract.html) for more details. diff --git a/docs/part-2/1_xds-server/3_build.md b/docs/part-2/1_xds-server/3_build.md new file mode 100644 index 0000000..8be3b33 --- /dev/null +++ b/docs/part-2/1_xds-server/3_build.md @@ -0,0 +1,73 @@ +# Build xds-server from scratch + +## Dependencies + +Install [Go](https://golang.org/doc/install), [npm](https://www.npmjs.com/), +[nodejs](https://nodejs.org/en/) and some other tools. + +Refer to [Prerequisites chapter](../1_Prerequisites.html) for more details. + +## Building + +### Native build + +Create a GOPATH variable(must be a full path): + +```bash +export GOPATH=$(realpath ~/workspace_go) +``` + +Clone this repo into your `$GOPATH/src/gerrit.automotivelinux.org/gerrit/src/xds` and use delivered Makefile: + +```bash +export GOPATH=${GOPATH}:${ROOTDIR} + +mkdir -p $ROOTDIR/src/gerrit.automotivelinux.org/gerrit/src/xds +cd $ROOTDIR/src/gerrit.automotivelinux.org/gerrit/src/xds +git clone https://gerrit.automotivelinux.org/gerrit/src/xds/xds-server +# or git clone ssh://YOUR_USERNAME@gerrit.automotivelinux.org:29418/src/xds/xds-server +cd xds-server +make all +``` + +And to install `xds-server` (by default in `/opt/AGL/xds/server`): + +```bash +make install +``` + +<!-- warning --> +**Warning:** + +Makefile install rule and default values in configuration file are set +to fit the docker setup. + +So you may need to adapt some settings when you want to install xds-server natively. +<!-- endwarning --> + +<!-- note --> +**Note:** + +Used `DESTDIR` to specify another install directory + +```bash +make install DESTDIR=$HOME/opt/xds-server +``` + +<!-- endnote --> + +### XDS docker image + +As an alternative to a pre-build image, you can rebuild the container from scratch. + +`xds-server` has been integrated as a flavour of AGL SDK docker image. + +So to rebuild docker image just execute following commands: + +```bash +# Clone docker-worker-generator git repo +git clone https://git.automotivelinux.org/AGL/docker-worker-generator +# Start build that will create a docker image +cd docker-worker-generator +make build FLAVOUR=xds +``` diff --git a/docs/part-2/1_xds-server/4_debug.md b/docs/part-2/1_xds-server/4_debug.md new file mode 100644 index 0000000..2b36697 --- /dev/null +++ b/docs/part-2/1_xds-server/4_debug.md @@ -0,0 +1,36 @@ +# Debugging + +## XDS server architecture + +The server part is written in *Go* and web app (basic HTML) in *Angular4*. + +```bash +| ++-- bin/ # where xds-server binary file will be built +| ++-- conf.d/ # Linux configuration and startup files (systemd user service) +| ++-- glide.yaml # Go package dependency file +| ++-- lib/ # sources of server part (Go) +| ++-- LICENSE # XDS server license +| ++-- main.go # main entry point of of Web server (Go) +| ++-- Makefile # makefile including +| ++-- README.md # +| ++-- scripts/ # hold various scripts used for installation or startup +| ++-- test/ # XDS test suite +| ++-- tools/ # temporary directory to hold development tools (like glide) +| ++-- vendor/ # temporary directory to hold Go dependencies packages +| ++-- webapp/ # source client basic web application +``` + +Visual Studio Code launcher settings can be found into `.vscode/launch.json`. |