path: root/doc/ApplicationGuide.md

**Sound Manager Application Guide**
====
<div align="right">Revision: 0.1</div>
<div align="right">TOYOTA MOTOR CORPORATION</div>
<div align="right">Advanced Driver Information Technology</div>
<div align="right">22th/Aug/2017</div>

* * *

## **<div id="Table\ of\ content">Table of content</div>**
- [Target reader of this document](#Target\ reader\ of\ this\ document)
- [Overview](#Overview)
- [Getting Start](#Getting\ Start)
	- [Supported environment](#Supported\ environment)
	- [Build](#Build)
	- [Configuring](#Configuring)
	- [How to call Sound Manager's APIs from your Application?](#How\ to\ call\ Sound\ Manager\ APIs\ from\ your\ Application?)
- [Supported usecase](#Supported\ usecase)
- [Software Architecture](#Software\ Architecture)
- [API reference](#API\ reference)
	- [CommandReceiver API](#CommandReceiver\ API)
	- [CommandSender API](#CommandSender\ API)
	- [CAmRoutingReceiver API](#CAmRoutingReceiver\ API)
	- [CAmRoutingSender API](#CAmRoutingSender\ API)
	- [Sound Manager Specific API](#Sound\ Manager\ Specific\ API)
- [Sequence](#Sequence)
	- [StartUp](#StartUp)
	- [Registration](#Registration)
	- [Request Sound Right](#Request\ Sound\ Right)
	- [Connect Sound Route](#Connect\ Sound\ Route)
	- [Start Sound Streaming](#Start\ Sound\ Streaming)
	- [Stop Sound Streaming](#Stop\ Sound\ Streaming)
	- [Disconnect Sound Route](#Disconnect\ Sound\ Route)
	- [Change Volume](#Change\ Volume)
	- [Set Mute State](#Set\ Mute\ State)
	- [Release Sound Right](#Release\ Sound\ Right)
	- [Audio Domain](#Audio\ Domain)
- [Sample code](#Sample\ code)
- [Limitation](#Limitation)
- [Next Plan](#Next\ Plan)

* * *

## **<div id="Target\ reader\ of\ this\ document">Target reader of this document</div>**
Application developer whose software uses sound output.

* * *

## **<div id="Overview">Overview</div>**
The sound manager is the service which provides **sound-right management** for multiple sound sources.  
This service based on GENIVI Audio Manager, and this package contains service binder and library for API calling.  
The reason why this service based on GENIVI Audio Manager is because the sound manager supports highly strong and flexible sound-right management function.

In order to understand, the below figure shows the one of typical usecases.
In this example, there are four sound mode.
1. Audio Off
2. Media Player
3. Tel (Ring and talking)
4. TTS (Text To Speech; typically it's used by Navigation sound)
![Figure: Typical usecase](parts/typical-usecase.png)

The important points are:
- **There is a priority for each sound source.**  
	In this example, "Tel" and "TTS" is stronger than "MediaPlayer". Therefore when the system got incoming call, all four outputs of MediaPlayer are muted automatically by Sound Manager. And in this timing, Sound Manager will issue the event to Media Player, then Media Player can stop the music. (Because depends on OEM's requirement, "Stop" is required, not "Mute".)  
    "Tel" and "TTS" have the same priority. So if TTS event happened on talking, each sound will output from independent speaker.  
    If on-hook button is touched, Sound Manager will resume previous sound mode. In this example, basically it's MediaPlayer sound. But if TTS still playing, three speaker will output MediaPlayer sound but one speaker will continue to output TTS sound.
- **Sound mode transition should be done by Sound Manager not Applications.**  
	Actually application cannot recognize all sound source and its priority, so some centerized manager is required. Sound Manager provides this function. Sound Manager has a database for usecase and priority and in line with this policy Sound Manager controls proper sound mode.


The below links show the example of Sound/Window mode transition.
* [Single window application](Display_Audio_Transition1.md)  
	This transition assumes target IVI system support only one window on screen. It's a similar transition to CES2017 demo.
* [Dual window application](Display_Audio_Transition2.md)  
	This transition assumes target IVI system support two window (split screen) on screen.

	Of course user can customize shortcut menu, but since it's too many states so this example limits shortcut menu as "Home", "MediaPlayer", "HVAC" and "Navigation".

* * *

## **<div id="Getting\ Start">Getting Start</div>**

### **<div id="Supported\ environment">Supported environment</div>**
| Item        | Description                       |
|:------------|:----------------------------------|
| AGL version | Daring Dab                        |
| Hardware    | Renesas R-Car Starter Kit Pro(M3) |


### **<div id="Build">Build</div>**

You can make Sound Manager object files by the following two stage operations.

**Download recipe**
If repo is already done, please start with git clone
```
$ mkdir WORK
$ cd WORK
$ repo init -b dab -m dab_4.0.0_xml -u https://gerrit.automotivelinux.org/gerrit/AGL/AGL-repo
$ repo sync
$ git clone git clone https://gerrit.automotivelinux.org/gerrit/staging/meta-hmi-framework

```

Then you can get the following recipe.
* `meta-hmi-framework/soundmanager`


**Bitbake**
```
$ source meta-agl/scripts/aglsetup.sh -m m3ulcb agl-demo agl-devel agl-appfw-smack
$ bitbake soundmanager
```


* * *

### **<div id="Configuring">Configuring</div>**
To use Sound Manager API, an application shall paste the following configuration definition into "config.xml" of application.
```
<feature name="urn:AGL:widget:required-api">
	<param name="soundmanager" value="ws" />
</feature>
```

* * *

### **<div id="How\ to\ call\ Sound\ Manager\ APIs\ from\ your\ Application?">How to call Sound Manager APIs from your Application?</div>**
Sound Manager provides a library which is called "libsoundmanager".  
This library treats "json format" as API calling.  
For example, if an application wants to call "connect()" API, the you should implement as below.  

At first the application should create the instance of libsoundmanager.
```
LibSoundmanager* libsm;
libsm = new LibSoundmanager(port, token);
```
The port and token is provided by Application Framework

Then assign the argument to JSON object
```
struct json_object* jobj = json_object_new_object();

json_object_object_add(jobj, "sourceID", json_object_new_int(100));
json_object_object_add(jobj, "sinkID", json_object_new_int(100));

```


And finally execute the "cal()" function.
```
libsm->call("connect", jobj);
```

Regarding the detail of connect() API, please refer [this](#CommandReceiver\ API) section.  
The first parameter is the name of API, so in this case "connect" is proper string.  
And the second parameter corresponds to arguments of "connect()" API.  



See also our [Sample code](#Sample\ code).


<br />

* * *

## **<div id="Supported\ usecase">Supported usecase</div>**
1. Active source change
	- When user choose different audio source with current one, IVI system stop or pause current source and activate new one.
	- When user connect external device e.g. iPhone, USB memory IVI system change active source automatically to connected one.
2. Active source locking
	- When user is in phone call, IVI restrict to change active source.
3. Interrupt source mixing
	- When car close to cross road IVI system reduce the volume of current source and mix with interrupt source e.g. Navigation Guidance.
4. Volume change
	- User can change the volume of active source or sink.
	- When user change volume during interruption e.g. Navigation Guidance, IVI system change its volume temporary or permanently.
5. Mute/unmute
	- User can mute/unmute current active source.
6. Volume management
	- When user change active source, IVI system mute/unmute to avoid distortion of sound.
7. Volume acceleration
	- When road noise is increased by speed, IVI system automatically change the volume of active source.
8. Routing sound
	- System needs to route sound stream to proper zones. (driver zone, passenger zone, rear seat zone)

[See also this page](https://wiki.automotivelinux.org/eg-ui-graphics-req-audiorouting)

* * *

## **<div id="Software\ Architecture">Software Architecture</div>**
The architecture of Sound Manager is shown below.  
Sound Manager is the service designed to be used by multiple applications.  
Therefore Sound Manager framework consists on two binder layers. Please refer the following figure.  
The upper binder is for application side security context for applications. The lower binder is for servide side security context.  
Usually application side binder has some business logic for each application, so the number of binders depend on the number of applications which use Sound Manager.  
On the other hand, regarding lower binder there is only one module in the system. This binder receives all messages from multiple applications (in detail, it comes from upper layer binder).

The communication protocols between libsoundmanager and upper binder, upper binder and lower binder, lower binder (soundmanager-binding) and AudioManager are WebSocket.

![software-stack.png](parts/software-stack.png)

* * *

## **<div id="API\ reference">API reference</div>**
"libsoundmanager" and "soundmanager_binding" provides several kinds of APIs, and these APIs basically correspond to GENIVI Audio Manager API. (Some APIs are Sound Manager original functions.)

For understanding, GENIVI Audio Manager stands for one core module and three plug-ins.
1. AudioManagerDaemon  
	This is a core module of Audio Manager.
2. AudioManagerCommandPlugin  
	This is a command interface for Audio Manager.
3. AudioManagerController  
	This plug-in can be used for sound-right management.
4. AudioManagerRountingPlugin  
	This plug-in abstracts the hardware and software. And sometimes there may be multiple plug-ins.

*) [See also GENIVI AudioManager Components](http://docs.projects.genivi.org/AudioManager/audiomanagercomponentspage.html)

![See also GENIVI AudioManager Components](parts/am-component.png)
(This figure was copied from GENIVI Web page.)

### **<div id="CommandReceiver\ API">CommandReceiver API</div>**
- [connect (const am_sourceID_t sourceID, const am_sinkID_t sinkID, am_mainConnectionID_t &mainConnectionID)](http://docs.projects.genivi.org/AudioManager/a00033.html#a62d8f5aee1e601d59f993c5a5561e234)
- [disconnect (const am_mainConnectionID_t mainConnectionID)](http://docs.projects.genivi.org/AudioManager/a00033.html#aa24d0146f4e3c75e02d6c0152e246da1)
- [setVolume (const am_sinkID_t sinkID, const am_mainVolume_t volume)](http://docs.projects.genivi.org/AudioManager/a00033.html#a6d47bc67473d75495260abe8c666fc7e)
- [volumeStep (const am_sinkID_t sinkID, const int16_t volumeStep)](http://docs.projects.genivi.org/AudioManager/a00033.html#ad7a4c1fe5a2ecfaae5484a14d8820e58)
- [setSinkMuteState (const am_sinkID_t sinkID, const am_MuteState_e muteState)](http://docs.projects.genivi.org/AudioManager/a00033.html#afae22041843c5349be16a6593d3ebb9c)
- [getListMainConnections (std::vector< am_MainConnectionType_s > &listConnections)](http://docs.projects.genivi.org/AudioManager/a00033.html#a59d10a7178e3227d0b8f415308c71179)

### **<div id="CommandSender\ API">CommandSender API</div>**
These APIs are callback function from GENIVI Audio Manager point of view, but it's treated as asynchronous "event" from Application point of view.  
To receive an event, Application should call "subscribe()" API, and register the event name.
Each CommandSender API corresponds to event name. For example, if an application wants to get the event of "cbNewMainConnection()", the event name shall be specified as "newMainConnection".
- [cbNewMainConnection (const am_MainConnectionType_s mainConnection)](http://docs.projects.genivi.org/AudioManager/a00034.html#a69ada9e19c65c1d078d8a5f473d08586)
- [cbRemovedMainConnection (const am_mainConnectionID_t mainConnection)](http://docs.projects.genivi.org/AudioManager/a00034.html#aa3b5906bcf682cff155fb24d402efd89)
- [cbMainConnectionStateChanged (const am_mainConnectionID_t connectionID, const am_ConnectionState_e connectionState)](http://docs.projects.genivi.org/AudioManager/a00034.html#a32aa8ab84632805a876e023a7aead810)
- [cbVolumeChanged (const am_sinkID_t sinkID, const am_mainVolume_t volume)](http://docs.projects.genivi.org/AudioManager/a00034.html#a4494fdd835137e572f2cf4a3aceb6ae5)
- [cbSinkMuteStateChanged (const am_sinkID_t sinkID, const am_MuteState_e muteState)](http://docs.projects.genivi.org/AudioManager/a00034.html#a6068ce59089fbdc63aec81e778aba238)

### **<div id="CAmRoutingReceiver\ API">CAmRoutingReceiver API</div>**
- [confirmRoutingReady (const uint16_t handle, const am_Error_e error)](http://docs.projects.genivi.org/AudioManager/a00053.html#a1dd1b89cccffeaafb1a3c11cebd7e48c)
- [registerSource (const am_Source_s &sourceData, am_sourceID_t &sourceID)](http://docs.projects.genivi.org/AudioManager/a00053.html#acadce23459d94cec496d17700cbde230)
- [registerDomain (const am_Domain_s &domainData, am_domainID_t &domainID)](http://docs.projects.genivi.org/AudioManager/a00053.html#a34841797b481e774867ce0a1efacd5f2)
- [ackConnect (const am_Handle_s handle, const am_connectionID_t connectionID, const am_Error_e error)](http://docs.projects.genivi.org/AudioManager/a00053.html#ad680eddb5bf7aa480308807903dcb592)
- [ackSetSourceState (const am_Handle_s handle, const am_Error_e error)](http://docs.projects.genivi.org/AudioManager/a00053.html#a11f6b0378a50296a72107d6a1fa7ec21)
- [ackDisconnect (const am_Handle_s handle, const am_connectionID_t connectionID, const am_Error_e error)](http://docs.projects.genivi.org/AudioManager/a00053.html#af478e5deb2e71e94c28cec497ac48ff4)

### **<div id="CAmRoutingSender\ API">CAmRoutingSender API</div>**
These APIs are also treaded as asynchronous event.  

- [setRoutingReady ()](http://docs.projects.genivi.org/AudioManager/a00054.html#a7a4d410e30df0e8240d25a57e3c72c6b)
- [asyncConnect (am_Handle_s &handle, am_connectionID_t &connectionID, const am_sourceID_t sourceID, const am_sinkID_t sinkID, const am_CustomConnectionFormat_t connectionFormat)](http://docs.projects.genivi.org/AudioManager/a00054.html#a8a81297be9c64511e27d85444c59b0d6)
- [asyncSetSourceState (am_Handle_s &handle, const am_sourceID_t sourceID, const am_SourceState_e state)](http://docs.projects.genivi.org/AudioManager/a00054.html#ab02d93d54ee9cd98776a3f2d274ee24d)
- [asyncDisconnect (am_Handle_s &handle, const am_connectionID_t connectionID)](http://docs.projects.genivi.org/AudioManager/a00054.html#a93ae95515730eb615ab5dfc1316d7862)

### **<div id="Sound\ Manager\ Specific\ API">Sound Manager Specific API</div>**
- [LibSoundmanager ()](api-ref/html/class_lib_soundmanager.html#a8b51e9891813cb62dd12109c017ad106)
- [call (const string& verb, struct json_object* arg)](api-ref/html/class_lib_soundmanager.html#a1fe952a4dabbab6126cc23e36c79c773)
- [call (const char* verb, struct json_object* arg)](api-ref/html/class_lib_soundmanager.html#a1fe952a4dabbab6126cc23e36c79c773)
- [subscribe (const string& event_name)](api-ref/html/class_lib_soundmanager.html#a9cd7c5470cb135f9b1aa56d790c7e91e)
- [unsubscribe (const string& event_name)](api-ref/html/class_lib_soundmanager.html#a21060844aa7efad6473b6104546afb06)
- [register_callback( void (\*event_cb)(const std::string& event, struct json_object\* event_contents), void (\*reply_cb)(struct json_object\* reply_contents))](api-ref/html/class_lib_soundmanager.html#a560edf9ae3b1e367ad4cbb31c7021d74)
- [run_eventloop()](api-ref/html/class_lib_soundmanager.html#abe71d3531e7888f47185a601b284e729)
* * *

## **<div id="Sequence">Sequence</div>**
### **<div id="StartUp">StartUp</div>**
![seq_startup.png](parts/seq_startup.svg)

### **<div id="Registration">Registration</div>**
![seq_registration.png](parts/seq_registration.svg)

### **<div id="Request\ Sound\ Right">Request Sound Right</div>**
![seq_requestsoundmode.png](parts/seq_requestsoundmode.svg)

### **<div id="Connect\ Sound\ Route">Connect Sound Route</div>**
![seq_connectsoundroute.png](parts/seq_connectsoundroute.svg)

### **<div id="Start\ Sound\ Streaming">Start Sound Streaming</div>**
![seq_startsoundstreaming.png](parts/seq_startsoundstreaming.svg)

### **<div id="Stop\ Sound\ Streaming">Stop Sound Streaming</div>**
![seq_stopsoundstreaming.png](parts/seq_stopsoundstreaming.svg)

### **<div id="Disconnect\ Sound\ Route">Disconnect Sound Route</div>**
![seq_disconnectsoundroute.png](parts/seq_disconnectsoundroute.svg)

### **<div id="Change\ Volume">Change Volume</div>**
![seq_changevolume.png](parts/seq_changevolume.svg)

### **<div id="Set\ Mute\ State">Set Mute State</div>**
![seq_setmutestate.png](parts/seq_setmutestate.svg)

### **<div id="Release\ Sound\ Right">Release Sound Right</div>**
![seq_releasesoundmode.png](parts/seq_releasesoundmode.svg)

* * *

### **<div id="Audio\ Domain">Audio Domain</div>**

One of the most important concept of Audio Manager is Audio Domain.
To use GENIVI Audio Manager based system, it may be better to understand this concept.
The below document should bring good understanding.

[GENIVI Audio Manager: Generic Controller Plug-in](http://events.linuxfoundation.org/sites/events/files/slides/AGL_AMM_presentation_A01.pdf)

Although strongly recommended to read whole pages, but you can get quick understanding by page.10 to 14.


# **<div id="Sample\ code">Sample code</div>**
You can find sample implementation of Sound Manager as below.
* `soundmanager/sample/radio`  
* `soundmanager/sample/mediaplayer`  
* `soundmanager/sample/phone`  

This samples are based on QML application. But the library is written as C library so it is usable for other HMI framework.  
To use Sound Manager, at first libsoundmanager should be enabled. As this sample, please refer to "soundmanager/sample/radio/app/main.cpp".  
Then as the calling sample from QML, please refer "soundmanager/sample/radio/app/Radio.qml". The following code is found in this file.  
```
onClicked: {
	var JsonArg = JSON.stringify({sourceID:100, sinkID:100})
	smw.call("connect",JsonArg)
```

And also we provide the other sample as below.
* `soundmanager/sample/radio_qml`  

This is an alternative way against useing libsoundmanager.
This sample doesn't use libsoundmanager but calls API via WebSocket class which is provided by QML. How to implement this way is found in "soundmanager/sample/radio_qml/app/api/BindingSoundManager.qml".

# **<div id="Limitation">Limitation</div>**
* The following APIs can be called, but probably it doesn't work  
	* setVolume
	* volumeStep
* The following APIs cannot be called so far.  
	* getVolume
* The following events never happen.  
	* volumeChanged
* The following sample application is under developing
	* soundmanager/sample/phone

# **<div id="Next\ Plan">Next Plan</div>**
* Add new calling style for API  
Current Sound Manger expects all APIs call be called as JSON format.
But Sound Manager provide "function call" interface in next version. So it will be possible to call all APIs of Sound Manager not only JSON format but also function style.

* Support source name and sink name as abstract alias instead of SourceID/SinkID  
The CommandReciever APIs requires SourceID/SinkID assigned dynamically as arguments, but after supporting this feature, you can call these APIs using abstract bname such like "radio" or "speaker". It will provide more readable code.

```
struct json_object* jobj = json_object_new_object();

json_object_object_add(jobj, "sourceID", json_object_new_string("radio"));
json_object_object_add(jobj, "sinkID", json_object_new_string("speaker"));

```