diff options
Diffstat (limited to 'docs/part-2/1_xds-server.md')
| -rw-r--r-- | docs/part-2/1_xds-server.md | 468 |
1 files changed, 0 insertions, 468 deletions
diff --git a/docs/part-2/1_xds-server.md b/docs/part-2/1_xds-server.md deleted file mode 100644 index a45a8f0..0000000 --- a/docs/part-2/1_xds-server.md +++ /dev/null @@ -1,468 +0,0 @@ -# XDS - X(cross) Development System Server - -`xds-server` is a web server that allows user to remotely cross build applications. - -- The first goal is to provide a multi-platform cross development tool with near-zero installation. -- The second goal is to keep application sources locally (on user's machine). - - Make it compatible with existing IT policies (e.g. corporate backup or SCM). - - Let user to continue to work as usual. - - Use his favorite editor. - - keep performance while editing/browsing sources. - - Avoids manual operation - -This powerful and portable webserver (written in [Go](https://golang.org)) -exposes a REST interface over HTTP. - -`xds-server` uses [Syncthing](https://syncthing.net/) tool to synchronize -projects files from user machine to build server machine or container. - -`xds-server` is commonly running on a build server (within a container or not) -and [xds-agent](2_xds-agent.html) must run on the developer/user machine in order -to setup the following connection chain: - -```schema - developer/user machine | build server or container - ---------------------------|----------------------------- - xds-cli <---> xds-agent <-|-> xds-server -``` - -**SEE ALSO**: [xds-cli](https://gerrit.automotivelinux.org/gerrit/gitweb?p=src/xds/xds-cli.git), -a command-line tool that allows you to send commands to `xds-agent / xds-server` -and for example build your application from command-line or from your favorite -IDE (such as Netbeans or Visual Studio Code) through `xds-agent <=> xds-server`. - -## How to run - -`xds-server` has been designed to easily compile and debug -[AGL](https://www.automotivelinux.org/) applications. That's why `xds-server` has -been integrated into AGL SDK docker container. - ->**Note:** For more info about AGL SDK docker container, please refer to -[AGL SDK Quick Setup](http://docs.automotivelinux.org/docs/getting_started/en/dev/reference/setup-sdk-environment.html) - -### Start xds-server within the container - -See [Installation based on Docker container](../part-1/2_install-xds-server.html#installation-based-on-docker-container) -chapter. - -### Manually Start XDS server - -XDS server is started as a service by Systemd. - -```bash -/lib/systemd/system/xds-server.service -``` - -This Systemd service starts a bash script - -```bash -/opt/AGL/xds/server/xds-server-start.sh -``` - -If needed you can change default setting by defining specific environment -variables file - -```bash -ssh -t -p 2222 devel@localhost vim /etc/default/xds-server -``` - -For example to control log level, just set LOG_LEVEL env variable. - -knowing that supported *level* are: - -- panic -- fatal -- error -- warn -- info -- debug - -```bash -docker exec ${CONTAINER_NAME} bash -c "echo 'LOG_LEVEL=debug' >> /etc/default/xds-server" -ssh -p 2222 devel@localhost -- "systemctl --user restart xds-server" -``` - -### SDK cross-toolchain Management - -#### Setup to add support of a new SDK family - -<!-- note --> -**Optional step**: read this chapter only if you plan to add a new SDK family. -<!-- endnote --> - -`xds-server` dynamically detects supported SDKs by scanning sub-directories of -`sdkScriptsDir` directory (see [Configuration chapter](#configuration)). - -Each sub-directory (usually name is the same as the SDK family) of `sdkScriptsDir` -must contain a set of scripts that will be called by `xds-server` to managed SDKs -of a specific family. - -These scripts are: - -- `add`: used to add/install a new SDK -- `db-dump`: returned the list of available and installed SDKs (JSON format) -- `db-update`: update SDKs database -- `get-family-config`: returned SDK family configuration structure (JSON format) -- `get-sdk-info`: extract SDK info (JSON format) from a SDK file/tarball -- `remove`: used to remove an existing SDK - -For example, here 2 SDKs family (`agl` and `zephyr`) are defined: - -```bash -# > tree ./sdks/ -./sdks/ -├── agl -│ ├── add -│ ├── db-dump -│ ├── db-update -│ ├── get-family-config -│ ├── get-sdk-info -│ └── remove -├── README.md -└── zephyr - ├── add -│ ├── db-dump -│ ├── db-update -│ ├── get-family-config -│ ├── get-sdk-info - └── remove -``` - -On startup `xds-server` will call in order: - -- `sdks/*/get-family-config` to get configuration of each SDK family. -- `sdks/*/db-dump` scripts to get the initial list of available and installed SDKs. - -Please refer to `sdks/README.md` for more information about scripts definition -and to understand how to add support of a new SDK family. - -#### Install a new SDK from command line - -Used `sdks` command of `xds-cli` tool to managed SDKs. - -```bash -# List all available SDKs -xds-cli sdks ls -a - -List of available SDKs: -ID NAME STATUS VERSION ARCH -ec15afe0 AGL-release-eel-4.99.4-raspberrypi3 Not Installed 4.99.4 armv7vehf-neon-vfpv4 -944d2d5a AGL-snapshots-master-latest-intel-corei7-64 Not Installed 4.99.3+snapshot corei7-64 -cf3a4365 AGL-release-dab-4.0.2-qemux86-64 Not Installed 4.0.2 corei7-64 -d65fe750 AGL-release-eel-latest-qemux86-64 Not Installed 4.99.3 corei7-64 -a0ae663d poky-agl-corei7-64-3.99.1+snapshot Installed 3.99.1+snapshot corei7-64 -87f0400b AGL-release-dab-3.99.3-m3ulcb-nogfx Installed 3.99.3 aarch64 -8c2f2841 AGL-release-dab-4.0.2-dragonboard-410c Not Installed 4.0.2 aarch64 -... - -# Install a SDK (downloaded + install) -xds-cli sdks install d65fe750 - -Installation of 'AGL-release-eel-latest-qemux86-64' SDK successfully started. -Downloading poky-agl-glibc-x86_64-agl-demo-platform-crosssdk-corei7-64-toolchain-4.99.5.sh ... ---2018-01-02 11:22:23-- https://download.automotivelinux.org/AGL/release/eel/latest/qemux86-64/deploy/sdk/poky-agl-glibc-x86_64-agl-demo-platform-crosssdk-corei7-64-toolchain-4.99.5.sh -Resolving download.automotivelinux.org (download.automotivelinux.org)... 199.19.213.77 -Connecting to download.automotivelinux.org (download.automotivelinux.org)|199.19.213.77|:443... connected. -HTTP request sent, awaiting response... 200 OK -Length: 665996704 (635M) [application/x-sh] -Saving to: ‘/tmp/tmp.wuQzLdImCS/poky-agl-glibc-x86_64-agl-demo-platform-crosssdk-corei7-64-toolchain-4.99.5.sh’ - - 0K .......... .......... .......... .......... .......... 0% 82,7K 2h11m - 50K .......... .......... .......... .......... .......... 0% 89,1K 2h6m - 100K .......... .......... .......... .......... .......... 0% 82,0K 2h8m -... - 50300K .......... .......... .......... .......... .......... 99% 2,15M 0s - 650350K .......... .......... .......... ....... 100% 4,04M=10m35s - -2018-01-02 12:17:06 (1024 KB/s) - ‘/tmp/tmp.CWyEj3z76Q/poky-agl-glibc-x86_64-agl-demo-platform-crosssdk-corei7-64-toolchain-4.99.5.sh’ saved [665996704/665996704] - -Automotive Grade Linux SDK installer version 5.0.0 -=================================================== -You are about to install the SDK to "/xdt/sdk/poky-agl/5.0.0/corei7-64". Proceed[Y/n]? Y -Extracting SDK....................................................................................................................done -Setting it up...done -SDK has been successfully set up and is ready to be used. -Each time you wish to use the SDK in a new shell session, you need to source the environment setup script e.g. - $ . /xdt/sdk/poky-agl/5.0.0/corei7-64/environment-setup-corei7-64-agl-linux - -SDK ID d65fe750-d3a7-38f5-83d8-3d3806054f8d successfully installed. - - -# Abort an installation that is in progress -xds-cli sdks abort -id d65fe750 - -# Install a SDK (using a local SDK package/file) -xds-cli sdks install --file $HOME/xds-workspace/sdks/poky-agl-glibc-x86_64-agl-demo-platform-crosssdk-corei7-64-toolchain-5.0.0.sh - -``` - -<!-- note --> -Installation based on a local SDK package is only supported when SDK file package -is located in `$HOME/xds-workspace/sdks` directory -<!-- endnote --> - -#### Install a new SDK from XDS Dashboard - -Open XDS-Dashboard in web-browser and select `SDKs` entry in left side menu. - -Then switch to `SDKs MANAGEMENT` view : - -{:: style="margin:auto; display:flex"} - -Use filter boxes to find the SDK you want to install and then click on plus -icon (Actions column) to start installation. - -SDK download and installation process may take several minutes and output of installation script (output of `add` script mentioned in above chapter). - -{:: style="margin:auto; display:flex"} - -Note that you can abort installation by clicking on `CANCEL` button. - -#### Un-install a SDK from command line - -Used `sdks` command of `xds-cli` tool to managed SDKs. - -```bash -# List installed SDKs -xds-cli sdks ls -List of installed SDKs: - ID NAME STATUS VERSION ARCH - c39e5998 poky-agl_aarch64_4.0.1 Installed 4.0.1 aarch64 - d610bfbf poky-agl-aarch64.current_on_iotbzh_download-3.99.1+snapshot Installed 3.99.1+snapshot aarch64.current_on_iotbzh_download - a0ae663d poky-agl-corei7-64-3.99.1+snapshot Installed 3.99.1+snapshot corei7-64 - 87f0400b AGL-release-dab-3.99.3-m3ulcb-nogfx Installed 3.99.3 aarch64 - 352c0584 poky-agl-corei7-64-3.99.2+snapshot Installed 3.99.2+snapshot corei7-64 - d65fe750 AGL-release-eel-latest-qemux86-64 Installed 4.99.5 corei7-64 - -# Un-install a SDK -xds-cli sdks uninstall d65fe750 -SDK ID d65fe750-d3a7-38f5-83d8-3d3806054f8d successfully deleted. -``` - -#### Un-install a SDK from XDS Dashboard - -Open XDS-Dashboard in web-browser and select `SDKs` entry in left side menu. - -If needed, switch to `BASIC SDKS VIEW` view and click on trash icon located -in the top-right corner of SDK card. - -### XDS server REST API and Web application - -`xds-server` exposes a REST API and serves a basic web-application. - -REST API based url is `http://localhost:8000/api/v1/` when XDS server is -running on your host (localhost) and basic web-application is available at -[http://localhost:8000](http://localhost:8000). - -Just replace `localhost` by the host name or ip when `xds-server` is running -on another host. - -```bash -# Get version using REST API -curl http://localhost:8000/api/v1/version - -# Open browser and local xds-server web-application -xdg-open http://localhost:8000 -``` - -Then follow instructions provided on this page to install and start `xds-agent` -that must run locally on your machine. - -See also [xds-agent documentation](2_xds-agent.html) for more details. - -## Build xds-server from scratch - -### Dependencies - -Install [Go](https://golang.org/doc/install), [npm](https://www.npmjs.com/), -[nodejs](https://nodejs.org/en/) and some other tools. -Refer to [Prerequisites chapter](./0_Abstract.html#prerequisites) for more details. - -### Building - -#### Native build - -Create a GOPATH variable(must be a full path): - -```bash -export GOPATH=$(realpath ~/workspace_go) -``` - -Clone this repo into your `$GOPATH/src/gerrit.automotivelinux.org/gerrit/src/xds` and use delivered Makefile: - -```bash -export GOPATH=${GOPATH}:${ROOTDIR} - -mkdir -p $ROOTDIR/src/gerrit.automotivelinux.org/gerrit/src/xds -cd $ROOTDIR/src/gerrit.automotivelinux.org/gerrit/src/xds -git clone https://gerrit.automotivelinux.org/gerrit/src/xds/xds-server -# or git clone ssh://YOUR_USERNAME@gerrit.automotivelinux.org:29418/src/xds/xds-server -cd xds-server -make all -``` - -And to install `xds-server` (by default in `/opt/AGL/xds/server`): - -```bash -make install -``` - -<!-- warning --> -**Warning:** - -Makefile install rule and default values in configuration file are set -to fit the docker setup. - -So you may need to adapt some settings when you want to install xds-server natively. -<!-- endwarning --> - -<!-- note --> -**Note:** - -Used `DESTDIR` to specify another install directory - -```bash -make install DESTDIR=$HOME/opt/xds-server -``` - -<!-- endnote --> - -#### XDS docker image - -As an alternative to a pre-build image, you can rebuild the container from scratch. - -`xds-server` has been integrated as a flavour of AGL SDK docker image. - -So to rebuild docker image just execute following commands: - -```bash -# Clone docker-worker-generator git repo -git clone https://git.automotivelinux.org/AGL/docker-worker-generator -# Start build that will create a docker image -cd docker-worker-generator -make build FLAVOUR=xds -``` - -### Configuration - -`xds-server` configuration is driven by a JSON config file (`server-config.json`). - -Here is the logic to determine which `server-config.json` file will be used: - -1. from command line option: `--config myConfig.json` -1. `$HOME/.xds/server/server-config.json` file -1. `/etc/xds/server/server-config.json` file -1. `<xds-server executable dir>/server-config.json` file - -Supported fields in configuration file are: - -- **httpPort** : HTTP port of client webapp/REST API -- **webAppDir** : location of client web application (default: webapp/dist) -- **shareRootDir** : root directory where projects will be copied -- **logsDir** : directory to store logs (eg. syncthing output) -- **sdkScriptsDir** : directory where scripts, used to managed SDKs, are installed -- **syncthing.binDir** : syncthing binaries directory (default: executable directory) -- **syncthing.home"** : syncthing home directory (usually .../syncthing-config) -- **syncthing.gui-address** : syncthing gui url (default <http://localhost:8385>) -- **syncthing.gui-apikey** : syncthing api-key to use (default auto-generated) - -All fields are optional and example below corresponds to the default values. - -```json -{ - "httpPort": 8000, - "webAppDir": "webapp/dist", - "shareRootDir": "${HOME}/.xds/server/projects", - "logsDir": "/tmp/logs", - "sdkScriptsDir": "${EXEPATH}/scripts/sdks", - "syncthing": { - "binDir": "./bin", - "home": "${HOME}/.xds/server/syncthing-config", - "gui-address": "http://localhost:8385", - "gui-apikey": "123456789", - } -} -``` - ->**Notes:** -> ->Environment variables are supported by using `${MY_VAR}` syntax. -> - -When `xds-server` is started as a systemd service, default environment variables -are set into `/etc/default/xds-server` file. - -`xds-server` configuration is also driven by a JSON config file (`server-config.json`), -and default JSON config is `/etc/xds/server/server-config.json`. - -<!-- note --> -**Note:** -You can use your own JSON config by settings `APP_CONFIG` variable of -`/etc/default/xds-server` file to your file, for example `/home/MYUSER/.xds/server/server-config.json` -<!-- endnote --> - -#### Disable syncthing - -`CloudSync` synchronization type based on `syncthing` tool can be disabled by -simply removing (or renaming) `"syncthing"` key in configuration file. -Here is a JSON configuration file example where syncthing key as been renamed: - -```json -{ - "httpPort": 8000, - "webAppDir": "webapp/dist", - "shareRootDir": "${HOME}/.xds/server/projects", - "logsDir": "/tmp/logs", - "sdkScriptsDir": "${EXEPATH}/scripts/sdks", - "syncthing_DISABLE": { - "binDir": "./bin", - "home": "${HOME}/.xds/server/syncthing-config", - } -} -``` - -On benefit to do that is to increase XDS-Server startup time. - -<!-- note --> -**Note:** - -- `CloudSync` (AKA syncthing) synchronzation type can also be disabled when `"syncthing"` key is not defined in JSON configuration file. - -<!-- endnote --> - -## Debugging - -### XDS server architecture - -The server part is written in *Go* and web app (basic HTML) in *Angular4*. - -```bash -| -+-- bin/ # where xds-server binary file will be built -| -+-- conf.d/ # Linux configuration and startup files (systemd user service) -| -+-- glide.yaml # Go package dependency file -| -+-- lib/ # sources of server part (Go) -| -+-- LICENSE # XDS server license -| -+-- main.go # main entry point of of Web server (Go) -| -+-- Makefile # makefile including -| -+-- README.md # -| -+-- scripts/ # hold various scripts used for installation or startup -| -+-- test/ # XDS test suite -| -+-- tools/ # temporary directory to hold development tools (like glide) -| -+-- vendor/ # temporary directory to hold Go dependencies packages -| -+-- webapp/ # source client basic web application -``` - -Visual Studio Code launcher settings can be found into `.vscode/launch.json`. |