docs: Improve documentation

Author: ErikKalkoken

README.md

@@ -8,7 +8,8 @@ A Python log handler for VictoriaLogs.
 [![codecov](https://codecov.io/gh/ErikKalkoken/python-victorialogs-handler/graph/badge.svg?token=2pPb3lid2k)](https://codecov.io/gh/ErikKalkoken/python-victorialogs-handler)
 [![license](https://img.shields.io/badge/license-MIT-green)](https://gitlab.com/ErikKalkoken/python-victorialogs-handler/-/blob/master/LICENSE)
 
-STATUS: In development
+> [!IMPORTANT]
+> STATUS: In development. The API is not yet stable.
 
 ## Description
 
@@ -18,7 +19,7 @@ STATUS: In development
 
 - **Asynchronous Processing**: Log events are queued and dispatched in a dedicated background thread, ensuring your application's main execution flow remains non-blocking and highly responsive.
 
-- **Efficient Request Batching**: To optimize network throughput and reduce overhead on your VictoriaLogs server, multiple log events are automatically combined into single-request batches.
+- **Efficient Request Batching**: To optimize network throughput and reduce overhead on your VictoriaLogs server, multiple log events are combined into chunks per request.
 
 - **Rich Exception Handling**: Automatically captures and flattens stack traces. Log messages include both the exception name and the full traceback as searchable fields.
 
@@ -36,14 +37,53 @@ pip install victorialogs-handler
 
 Then it can be used like any other logging handler to configure a logger.
 
-## Usage
+## Quick start
 
-Please see the directory `/examples` for examples on how to use the handler.
+> [!NOTE]
+> The script assumes that there is a VictoriaLogs server running
+> on the same system at the default URL: `http://localhost:9428`
+
+Here is a quick example on how to use the handler in your Python script:
+
+```python
+import atexit
+import logging
+
+from vlogs_handler import VictoriaLogsHandler
+
+# Create a custom logger with INFO level
+logger = logging.getLogger(__name__)
+logger.setLevel(logging.INFO)
+
+# Add a handler for VictoriaLogs
+vlogs_handler = VictoriaLogsHandler()
+vlogs_handler.setLevel(logging.DEBUG)
+logger.addHandler(vlogs_handler)
+
+# Make sure to flush logs before exiting
+atexit.register(logging.shutdown)
+
+# Log example
+logger.info("This is an info message")
+```
+
+Please see the directory `/examples` for additional examples on how to use the handler.
 
 ## Technical details
 
 This section documents technical details of the solution.
 
+### Technical process
+
+The vlogs handler works as follows:
+
+1. When a a log event is received, it is converted into JSON and stored in the buffer
+2. At an interval (e.g. 5 second) or when a threshold is reached (e.g. 125 logs) a background worker starts the process of submitting logs from the buffer to the log server
+3. Logs are combined into chunks (e.g. 1.000 logs per request) and then submitted to the log server using vlog's the JSON Stream API
+4. In case submission fails (e.g. the log server is down) the logs will be stored back to the buffer for later retry
+5. In case the buffer is full new logs will be discarded.
+6. When the service shuts down, logs remaining in the buffer are transferred to the log server.
+
 ### LogRecord fields
 
 The following fields will be transferred for each log event. They are derived from Python's [LogRecord](https://docs.python.org/3/library/logging.html#logrecord-objects):

examples/basic_example.py

@@ -10,16 +10,9 @@
 
 from vlogs_handler import VictoriaLogsHandler
 
-# Create a custom logger
+# Create a custom logger with INFO level
 logger = logging.getLogger(__name__)
-logger.setLevel(logging.DEBUG)  # Set the lowest level to capture
-
-# Add a handler for console logging
-console_handler = logging.StreamHandler()
-console_handler.setLevel(logging.DEBUG)
-log_format = logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s")
-console_handler.setFormatter(log_format)
-logger.addHandler(console_handler)
+logger.setLevel(logging.INFO)
 
 # Add a handler for VictoriaLogs
 vlogs_handler = VictoriaLogsHandler()
@@ -29,11 +22,5 @@
 # Make sure to flush logs before exiting
 atexit.register(logging.shutdown)
 
-# Log example with structured data
-logger.info("basic_example: This is an info message", extra={"user_id": 42})
-
-# Log example with an exception
-try:
-    _ = 1 / 0
-except Exception:
-    logger.exception("basic_example: This is an exception")
+# Log example
+logger.info("This is an info message")

examples/extended_example.py

@@ -0,0 +1,39 @@
+"""
+This script demonstrates how to setup and use a logger with the vlogs handler.
+
+Note that the script assumes that there is a vlogs server running
+on the same system at the default URL.
+"""
+
+import atexit
+import logging
+
+from vlogs_handler import VictoriaLogsHandler
+
+# Create a custom logger
+logger = logging.getLogger(__name__)
+logger.setLevel(logging.DEBUG)  # Set the lowest level to capture
+
+# Add a handler for console logging
+console_handler = logging.StreamHandler()
+console_handler.setLevel(logging.DEBUG)
+log_format = logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s")
+console_handler.setFormatter(log_format)
+logger.addHandler(console_handler)
+
+# Add a handler for VictoriaLogs
+vlogs_handler = VictoriaLogsHandler()
+vlogs_handler.setLevel(logging.DEBUG)
+logger.addHandler(vlogs_handler)
+
+# Make sure to flush logs before exiting
+atexit.register(logging.shutdown)
+
+# Log example with structured data
+logger.info("basic_example: This is an info message", extra={"user_id": 42})
+
+# Log example with an exception
+try:
+    _ = 1 / 0
+except Exception:
+    logger.exception("basic_example: This is an exception")

src/vlogs_handler/handler.py

@@ -1,16 +1,4 @@
-"""Module handler provides the implementation of the vlogs handler.
-
-- The handler collects log events and sends them asynchronously to the vlogs server
-- Log records are converted into JSON objects.
-    The handler uses orjson for significantly better performance
-    and support of additional data types
-- The handler uses the vlogs's JSON Stream API for data ingestion.
-- Logs are submitted when the flush interval expires or when the batch size is exceeded
-- Multiple logs are chunked together into a single request using the ndjson protocol
-    to minimize the number of requests.
-- When submission to the log server fails, logs are returns into the buffer
-    for later retry.
-"""
+"""Module handler provides the implementation of the vlogs handler."""
 
 import io
 import logging
@@ -57,7 +45,8 @@
 
 
 class VictoriaLogsHandler(logging.Handler):
-    """VictoriaLogsHandler dispatches log events to a VictoriaLogs server.
+    """VictoriaLogsHandler is a standard log handler
+    that dispatches log events to a VictoriaLogs server.
 
     Args:
         batch_size: New logs are submitted immediately once this threshold is reached.