MDBF-1224: fix the structure of the plugin MD file

This requires moxygen 2.1.16! Added a check

Implemented a structure for the plugin API documentation.
On the top there's a main page to give top level overview and
concepts.
It eventually references the Moxygen documentation.
The moxygen documentation is a series of nested doxygen groups
that define the structure for the whole Plugin API documentation.

There are top levels like "server plugin API" and "client plugin API".
These branch into futher subgroups as needed.

Author: gkodinov

include/mysql/Doxyfile.generated_docs_plugin_api

@@ -9,4 +9,4 @@ RECURSIVE              = YES
 GENERATE_HTML          = NO
 GENERATE_LATEX         = NO
 GENERATE_XML           = YES
-PREDEFINED             = HAVE_PSI_SOCKET_INTERFACE HAVE_PSI_1 USE_PSI_1
+PREDEFINED             = HAVE_PSI_SOCKET_INTERFACE HAVE_PSI_1 USE_PSI_1 MYSQL_DYNAMIC_PLUGIN __cplusplus

include/mysql/auth_dialog_client.h

@@ -1,4 +1,3 @@
-#ifndef MYSQL_AUTH_DIALOG_CLIENT_INCLUDED
 /* Copyright (C) 2010 Sergei Golubchik and Monty Program Ab
 
    This program is free software; you can redistribute it and/or modify
@@ -14,15 +13,22 @@
    along with this program; if not, write to the Free Software
    Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1335 USA */
 
+#ifndef MYSQL_AUTH_DIALOG_CLIENT_INCLUDED
+#define MYSQL_AUTH_DIALOG_CLIENT_INCLUDED
+
 /**
   @file
 
   Definitions needed to use Dialog client authentication plugin
 */
 
+/**
+  @addtogroup plugin_client_api
+  @{
+ */
+
 struct st_mysql;
 
-#define MYSQL_AUTH_DIALOG_CLIENT_INCLUDED
 
 /**
   type of the mysql_authentication_dialog_ask function
@@ -53,4 +59,6 @@ typedef char *(*mysql_authentication_dialog_ask_t)(struct st_mysql *mysql,
 #define PASSWORD_QUESTION       "\4"
 #define LAST_PASSWORD           "\5"
 
+/** @} */
+
 #endif

include/mysql/client_plugin.h

@@ -1,4 +1,3 @@
-#ifndef MYSQL_CLIENT_PLUGIN_INCLUDED
 /* Copyright (C) 2010 Sergei Golubchik and Monty Program Ab
    Copyright (c) 2010, 2011, Oracle and/or its affiliates.
 
@@ -15,15 +14,21 @@
    along with this program; if not, write to the Free Software
    Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1335  USA */
 
+#ifndef MYSQL_CLIENT_PLUGIN_INCLUDED
+#define MYSQL_CLIENT_PLUGIN_INCLUDED
 /**
   @file
 
   MySQL Client Plugin API
 
   This file defines the API for plugins that work on the client side
 */
-#define MYSQL_CLIENT_PLUGIN_INCLUDED
 
+/** 
+  @defgroup plugin_client_api Client Plugin API
+  API for plugins that work on the client side
+  @{
+*/
 /*
   On Windows, exports from DLL need to be declared
   Also, plugin needs to be declared as extern "C" because MSVC 
@@ -185,5 +190,7 @@ mysql_client_register_plugin(struct st_mysql *mysql,
 **/
 int mysql_plugin_options(struct st_mysql_client_plugin *plugin,
                          const char *option, const void *value);
+
+/** @} */
 #endif
 

include/mysql/generate_plugin_api_docs.sh

@@ -10,7 +10,33 @@
 #   md/<group>.md - documentation for group <group>
 #   md/<page>.md - documentation for each <page>
 # Note: target_dir is created if not present.
-# Note: requires at least moxygen 2.1.11
+
+# compare version numbers
+# usage: vercmp <versionnr1> <versionnr2>
+#         with format for versions xxx.xxx.xxx
+# returns: 0 if versionnr1 equal or greater
+#          1 if versionnr1 lower
+
+vercmp()
+{
+  local a1 b1 c1 a2 b2 c2
+  v1=$1
+  v2=$2
+  set -- $( echo "$v1" | sed 's/\./ /g' )
+  a1=$1 b1=$2 c1=$3
+  set -- $( echo "$v2" | sed 's/\./ /g' )
+  a2=$1 b2=$2 c2=$3
+  ret=$(( (a1-a2)*1000000+(b1-b2)*1000+c1-c2 ))
+  if [ $ret -lt 0 ] ; then
+    v=-1
+  elif [ $ret -eq 0 ] ; then
+    v=0
+  else
+    v=1
+  fi
+  printf "%d" $v
+  return
+}
 
 # If no target_dir is specified, the current directory is used.
 if [ -n "$1" ]; then
@@ -22,6 +48,15 @@ fi
 # Exit on error, undefined variable, or pipe failure
 set -euo pipefail
 
+#check if moxygen version is good enough
+moxygen_version=$(moxygen --version)
+require_moxygen_version="2.1.16"
+if [ $(vercmp "$moxygen_version" "$require_moxygen_version") -lt 0 ]; then
+    echo "moxygen version $require_moxygen_version or higher is required," \
+         "but $moxygen_version is installed."
+    exit 1
+fi
+
 # Clean up the output directory, if it exists
 rm -rf "$TARGET_DIR/md"
 rm -rf "$TARGET_DIR/xml"

include/mysql/index.dox

@@ -1,11 +1,150 @@
 /**
   @mainpage Plugin API Documentation
 
-  The MariaDB plugin API allows development of plugins that can extend the basic server functionality.
+  MariaDB Server Plugin API is a C-based extensibility framework
+  (largely compatible with MySQL’s but extended) that allows dynamic or
+  static loading of modules to add or modify server behavior without
+  recompiling the core server.
 
-   - Concepts
+   - @subpage concepts
    - Get started: Building a plugin
    - Plugin types
    - Plugin services
    - [API reference](api.md#)
-*/
+*/
+
+/**
+  @page concepts Plugin API Concepts
+
+  @section concept_big_picture The Big Bicture: The server, Plugins and Interfaces
+
+  The server does, at certain stages of its execution, call plugin APIs when it
+  needs certain operation performed: e.g. authenticate a user, log a message,
+  read and write from/to a database table etc.
+
+  These API are like written functionality contracts.
+  They allow separating the server functionality from the actual API
+  implementations themselves. And, since the API implementions are a dynamic set,
+  adding new implementations extends the server functionality
+  (e.g. adds new ways a user can be authenticated or message be logged)
+  without having to re-compile and even re-start the server itself.
+
+  The MariaDB plugin infrastructure uses the following logical terms:
+
+  @subsection concept_plugin_api Plugin API
+
+  A well defined, versioned and stable interface specification.
+  It's also abstract, as it's not tied to any one specific implementation.
+
+  @subsection concept_plugin Plugin
+
+  A named and versioned implementation of a specific @ref concept_plugin_api. 
+
+  @subsection concept_plugin_library Plugin Library
+  
+  (habitually abbreviated as a plugin)
+  An executable binary that contains one or more @ref concept_plugin that can
+  be accessed by the server when it needs to call plugin API implementations.
+
+
+  @sa @ref implementation on how the above concepts are implemented.
+*/
+
+/**
+  @page implementation Plugin API Implementation
+
+  This is how the individual abstract concepts are implemented and
+  interact with each other.
+
+  @section impl_server The Server Plugin API implementation
+
+  The server is the bedrock of the plugin infrastructure and contains the
+  base implementation to make use of it.
+  Plugins can theoretically be loaded into other binaries, but chances are
+  that they won't operate all that well (if at all) in this case since they
+  heavily depend on server's infrastructure.
+
+  Here's what the server implements to support plugins:
+
+  @subsection impl_server_global_plugin_list The global plugin list
+
+  The server maintains a global in-memory list of all available plugins (plugin_array), mirrored into a set of hash tables (plugin_hash) and guarded by LOCK_plugin.
+  
+  The plugin libraries containing the available plugins can be:
+   - statically linked into the server binary
+   - dynamically loaded at startup via the --plugin-load... command line arguments
+   - dynamically loaded just prior to the server's commencement of normal operations from the mysql.plugins table.
+   - dynamically loaded at runtime via the INSTALL PLUGIN SQL command.
+   - unloaded at runtime via the UNINSTALL PLUGIN SQL command
+   - unloaded at server shutdown 
+
+  The server implements all of the above possibilities in its code.
+
+  It also implements and provides a list of interfaces the plugins can use to call back and interact with the server.
+  These are called @ref impl_server_plugin_services.
+
+  @subsection impl_server_plugin_services Plugin Services Implementation
+
+  The server maintains a global named list of plugin service API implementations.
+  Every time a @ref concept_plugin_library is initialized, the server gets its global
+  service API implementation list and sets pointers to each individual API slot into
+  the empty pointer block provided by the @ref concept_plugin_library.
+  The library code compiled to call these API pointers can now access
+  the actual server implementations of these services API.
+
+  @sa @ref plugin_api_services for a list of the services provided by the server.
+
+  @subsection impl_server_plugin_api_call How the server calls a plugin API.
+
+  At certain opportune stages of server's normal operation the server might decide
+  to "reach out" to a plugin by calling a plugin API. Since the plugins are named it
+  can call any individual plugin by name. E.g. when executing `CREATE TABLE engine=foo`, 
+  the server will:
+
+  - look "foo" into the cache of plugin "references" (plugin_ref) for that plugin type, if available.
+  - If no cached reference is found:
+      - Take LOCK_plugin
+      - Look "foo" up into the global plugin_array for implementations of the specific plugin API (storage engine in this case)
+      - If found, create a reference (plugin_ref) to the plugin.
+  - call API "method" (function pointers) through the plugin_ref, if found.
+  - when done, either store the plugin_ref into a cache for reuse or release it.
+
+  @section impl_plugin_library The Plugin library Implementation
+
+  A @ref concept_plugin_library is executable code, either a part of
+  the server itself or bundled as a shared library.
+
+  @subsection impl_plugin_library_static Statically linked plugin libraries.
+
+  Plugins linked statically to the server are collected into a global
+  server "plugin library" and "installed" (and initialized) at server startup.
+
+  @subsection impl_plugin_library_dynamic Dynamic plugin libraries
+
+  A @ref concept_plugin_library is usually compiled as OS dynamic library:
+   DLLs on MS Windows, .so on Unixes. 
+  What differentiates a @ref concept_plugin_library from a regular OS
+  shared library is that it exposes a specific public symbol
+  _maria_plugin_declarations_ that the server recognizes. 
+  In fact it also exposes _maria_plugin_interface_version_ and
+  _maria_sizeof_struct_st_plugin_, but these are auxilary.
+
+  _maria_plugin_declarations_ is an array of @ref st_maria_plugin structures,
+  each describing a @ref concept_plugin this @ref concept_plugin_library provides.
+
+  When the @ref concept_plugin_library is loaded into the server:
+   * the library is version and maturity checked
+   * for each @ref concept_plugin contained
+     * plugin API version check is performed
+     * the plugin is initialized
+     * the plugin system and status variables are added to the variable global lists
+     * the plugin is added into the global plugin_array.
+
+  At unload time the effect of plugin library load above is reversed.  
+
+  @section impl_plugin Plugin implementation
+
+  A MariaDB plugin is a logical, versioned implementation of exactly two interfaces:
+   - The base @ref st_maria_plugin plugin management interface
+   - A specific plugin interface, one of @ref plugin_types.
+ */