Docs

git-svn-id: https://svn.kapsi.fi/jpa/nanopb@956 e3a754e5-d11d-0410-8d38-ebb782a927b9

7 files changed, 161 insertions, 45 deletions

diff --git a/docs/Makefile b/docs/Makefile
index e0ddc37d..d1d9587d 100644
--- a/docs/Makefile
+++ b/docs/Makefile
@@ -1,4 +1,4 @@
-all: index.html encoding.html concepts.html reference.html
+all: index.html concepts.html reference.html
 
 %.html: %.rst
 	rst2html --stylesheet=lsr.css --link-stylesheet $< $@
\ No newline at end of file
diff --git a/docs/concepts.rst b/docs/concepts.rst
index f2475aec..58620a85 100644
--- a/docs/concepts.rst
+++ b/docs/concepts.rst
@@ -2,12 +2,42 @@
 Nanopb: Basic concepts
 ======================
 
-The things outlined here are common to both the encoder and the decoder part.
+.. include :: menu.rst
 
-.. sectnum::
+The things outlined here are common to both the encoder and the decoder part.
 
 .. contents::
 
+Proto files
+===========
+All Protocol Buffers implementations use .proto files to describe the message format.
+The point of these files is to be a portable interface description language.
+
+Compiling .proto files for nanopb
+---------------------------------
+Nanopb uses the Google's protoc compiler to parse the .proto file, and then a python script to generate the C header and source code from it::
+
+    user@host:~$ protoc -omessage.pb message.proto
+    user@host:~$ python ../generator/nanopb_generator.py message.pb
+    Writing to message.h and message.c
+    user@host:~$
+
+Compiling .proto files with nanopb options
+------------------------------------------
+Nanopb defines two extensions for message fields described in .proto files: *max_size* and *max_count*.
+These are the maximum size of a string and maximum count of items in an array::
+
+    required string name = 1 [(nanopb).max_size = 40];
+    repeated PhoneNumber phone = 4 [(nanopb).max_count = 5];
+
+To use these extensions, you need to place an import statement in the beginning of the file::
+
+    import "nanopb.proto";
+
+This file, in turn, requires the file *google/protobuf/descriptor.proto*. This is usually installed under */usr/include*. Therefore, to compile a .proto file which uses options, use a protoc command similar to::
+
+    protoc -I/usr/include -Inanopb/generator -I. -omessage.pb message.proto
+
 Streams
 =======
 
@@ -130,21 +160,78 @@ required bytes data = 1 [(nanopb).max_size = 40];
 
 The maximum lengths are checked in runtime. If string/bytes/array exceeds the allocated length, *pb_decode* will return false. 
 
-For more information about callbacks, see the `Encoding` and `Decoding` sections.
+Field callbacks
+===============
+When a field has dynamic length, nanopb cannot statically allocate storage for it. Instead, it allows you to handle the field in whatever way you want, using a callback function.
+
+The `pb_callback_t`_ structure contains a function pointer and a *void* pointer you can use for passing data to the callback. The actual behavior of the callback function is different in encoding and decoding modes.
+
+.. _`pb_callback_t`: reference.html#pb-callback-t
+
+Encoding callbacks
+------------------
+::
+
+    bool (*encode)(pb_ostream_t *stream, const pb_field_t *field, const void *arg);
+
+When encoding, the callback should write out complete fields, including the wire type and field number tag. It can write as many or as few fields as it likes. For example, if you want to write out an array as *repeated* field, you should do it all in a single call.
+
+The callback may be called multiple times during a single call to `pb_encode`_. It must produce the same amount of data every time.
+
+.. _`pb_encode`: reference.html#pb-encode
+
+This callback writes out a dynamically sized string::
+
+    bool write_string(pb_ostream_t *stream, const pb_field_t *field, const void *arg)
+    {
+        char *str = get_string_from_somewhere();
+        if (!pb_encode_tag_for_field(stream, field))
+            return false;
+        
+        return pb_encode_string(stream, (uint8_t*)str, strlen(str));
+    }
+
+Decoding callbacks
+------------------
+::
+
+    bool (*decode)(pb_istream_t *stream, const pb_field_t *field, void *arg);
+
+When decoding, the callback receives a length-limited substring that reads the contents of a single field. The field tag has already been read.
+
+The callback will be called multiple times for repeated fields. For packed fields, you can either read multiple values until the stream ends, or leave it to `pb_decode`_ to call your function over and over until all values have been read.
+
+.. _`pb_decode`: reference.html#pb-decode
+
+This callback reads multiple integers and prints them::
+
+    bool read_ints(pb_istream_t *stream, const pb_field_t *field, void *arg)
+    {
+        while (stream.bytes_left)
+        {
+            uint64_t value;
+            if (!pb_decode_varint(stream, &value))
+                return false;
+            printf("%lld\n", value);
+        }
+        return true;
+    }
 
 Field description array
 =======================
 
 For using the *pb_encode* and *pb_decode* functions, you need an array of pb_field_t constants describing the structure you wish to encode. This description is usually autogenerated from .proto file.
 
-::
+For example this submessage in the Person.proto file::
 
- message PhoneNumber {
-    required string number = 1 [(nanopb).max_size = 40];
-    optional PhoneType type = 2 [default = HOME];
+ message Person {
+    message PhoneNumber {
+        required string number = 1 [(nanopb).max_size = 40];
+        optional PhoneType type = 2 [default = HOME];
+    }
  }
 
-::
+generates this field description array for the structure *Person_PhoneNumber*::
 
  const pb_field_t Person_PhoneNumber_fields[3] = {
     {1, PB_HTYPE_REQUIRED | PB_LTYPE_STRING,
@@ -160,8 +247,6 @@ For using the *pb_encode* and *pb_decode* functions, you need an array of pb_fie
     PB_LAST_FIELD
  };
 
-For more information about the format, see the `Generated code` section.
-
 
 Return values and error handling
 ================================
diff --git a/docs/encoding.rst b/docs/encoding.rst
deleted file mode 100644
index 3f673f35..00000000
--- a/docs/encoding.rst
+++ /dev/null
@@ -1,18 +0,0 @@
-=========================
-Nanopb: Encoding messages
-=========================
-
-The basic way to encode messages is to:
-
-1) Create an `output stream`_.
-2) Fill a structure with your data.
-3) Call *pb_encode* with the stream, a pointer to *const pb_field_t* array and a pointer to your structure.
-
-A few extra steps are necessary if you need to know the size of the message beforehand, or if you have dynamically sized fields.
-
-.. _`output stream`: concepts.html#output-streams
-
-Function: pb_encode
-===================
-
-
diff --git a/docs/index.rst b/docs/index.rst
index 0ae6f0fd..93f04c6c 100644
--- a/docs/index.rst
+++ b/docs/index.rst
@@ -2,6 +2,8 @@
 Nanopb: Protocol Buffers with small code size
 =============================================
 
+.. include :: menu.rst
+
 Nanopb is an ANSI-C library for encoding and decoding messages in Google's `Protocol Buffers`__ format with minimal requirements for RAM and code space.
 It is primarily suitable for 32-bit microcontrollers.
 
@@ -55,8 +57,12 @@ For starters, consider this simple message::
     required int32 value = 1;
  }
 
-Save this in *example.proto* and run it through *nanopb_generate.py*. You
-should now have in *example.h*::
+Save this in *example.proto* and compile it::
+
+    user@host:~$ protoc -omessage.pb message.proto
+    user@host:~$ python ../generator/nanopb_generator.py message.pb
+
+You should now have in *example.h*::
 
  typedef struct {
     int32_t value;
@@ -75,19 +81,8 @@ After that, buffer will contain the encoded message.
 The number of bytes in the message is stored in *stream.bytes_written*.
 You can feed the message to *protoc --decode=Example example.proto* to verify its validity.
 
-Library reference
-=================
-
-**Encoding**
-
-**Decoding**
-
-**Specifying field options**
-
-**Generated code**
-
 Wishlist
 ========
 #) A specialized encoder for encoding to a memory buffer. Should serialize in reverse order to avoid having to determine submessage size beforehand.
-#) A cleaner rewrite of the source generator.
+#) A cleaner rewrite of the Python-based source generator.
 #) Better performance for 16- and 8-bit platforms: use smaller datatypes where possible.
diff --git a/docs/lsr.css b/docs/lsr.css
index 81badb23..86053252 100644
--- a/docs/lsr.css
+++ b/docs/lsr.css
@@ -214,7 +214,7 @@ table.docutils th {
 }
 
 div.sidebar {
-  margin: 0em 2em 2em 0em;
+  margin: 2em 2em 2em 0em;
   padding: 0em 1em;
   border-top: 1px solid #aaa;
   border-left: 1px solid #aaa;
diff --git a/docs/menu.rst b/docs/menu.rst
new file mode 100644
index 00000000..4c643bc8
--- /dev/null
+++ b/docs/menu.rst
@@ -0,0 +1,10 @@
+.. sidebar :: Documentation index
+
+    1) `Overview`_
+    2) `Concepts`_
+    3) `API reference`_
+    
+.. _`Overview`: index.html
+.. _`Concepts`: concepts.html
+.. _`API reference`: reference.html
+
diff --git a/docs/reference.rst b/docs/reference.rst
index 4c8c874e..b8f9454e 100644
--- a/docs/reference.rst
+++ b/docs/reference.rst
@@ -2,6 +2,8 @@
 Nanopb: API reference
 =====================
 
+.. include :: menu.rst
+
 .. contents ::
 
 pb.h
@@ -42,7 +44,7 @@ PB_HTYPE_CALLBACK    0x30  A field with dynamic storage size, data is
 
 pb_field_t
 ----------
-Describes a single structure field with memory position in relation to others. ::
+Describes a single structure field with memory position in relation to others. The descriptions are usually autogenerated. ::
 
     typedef struct _pb_field_t pb_field_t;
     struct _pb_field_t {
@@ -63,6 +65,48 @@ Describes a single structure field with memory position in relation to others. :
 :array_size:    Maximum number of entries in an array, if it is an array type.
 :ptr:           Pointer to default value for optional fields, or to submessage description for PB_LTYPE_SUBMESSAGE.
 
+The *uint8_t* datatypes limit the maximum size of a single item to 255 bytes and arrays to 255 items. Compiler will warn "Initializer too large for type" if the limits are exceeded. The types can be changed to larger ones if necessary.
+
+pb_bytes_array_t
+----------------
+An byte array with a field for storing the length::
+
+    typedef struct {
+        size_t size;
+        uint8_t bytes[1];
+    } pb_bytes_array_t;
+
+In an actual array, the length of *bytes* may be different.
+
+pb_callback_t
+-------------
+Part of a message structure, for fields with type PB_HTYPE_CALLBACK::
+
+    typedef struct _pb_callback_t pb_callback_t;
+    struct _pb_callback_t {
+        union {
+            bool (*decode)(pb_istream_t *stream, const pb_field_t *field, void *arg);
+            bool (*encode)(pb_ostream_t *stream, const pb_field_t *field, const void *arg);
+        } funcs;
+        
+        void *arg;
+    };
+
+The *arg* is passed to the callback when calling. It can be used to store any information that the callback might need.
+
+When calling `pb_encode`_, *funcs.encode* must be set, and similarly when calling `pb_decode`_, *funcs.decode* must be set. The function pointers are stored in the same memory location but are of incompatible types.
+
+pb_wire_type_t
+--------------
+Protocol Buffers wire types. These are used with `pb_encode_tag`_. ::
+
+    typedef enum {
+        PB_WT_VARINT = 0,
+        PB_WT_64BIT  = 1,
+        PB_WT_STRING = 2,
+        PB_WT_32BIT  = 5
+    } pb_wire_type_t;
+
 pb_encode.h
 ===========