Skip to content

bouncer crash in metrics.lua #105

Description

@neilbags

Our crowdsec-nginx-bouncer crashed yesterday any did not recover until nginx was restarted:

2026/06/03 04:30:07 [error] 2779201#2779201: *5880858 lua entry thread aborted: runtime error: /usr/lib/crowdsec/lua/plugins/crowdsec/metrics.lua:137: bad argument #1 to 'decode' (string expected, got nil)
stack traceback:
coroutine 0:
        [C]: in function 'decode'
        /usr/lib/crowdsec/lua/plugins/crowdsec/metrics.lua:137: in function 'toJson'
        /usr/lib/crowdsec/lua/plugins/crowdsec/metrics.lua:153: in function 'sendMetrics'
        /usr/lib/crowdsec/lua/crowdsec.lua:325: in function </usr/lib/crowdsec/lua/crowdsec.lua:294>, context: ngx.timer

This was preceded by errors about the shared dict being full, however we have a lot of this in the logs without any crashes. The shared dict being full is a regression - the default cache size is too small (we use premium blocklists), we had previously increased its size, but this configuration was overwritten in a package update. I am not sure if this is related to the crash, but I figure that it is feasible that the cache being full may have caused some truncation, resulting in the crash.

I would expect that:

  1. If the bouncer crashes it should recover without needing to restart nginx

  2. A package update should not overwrite a configuration in /etc without a prompt

I understand from previous correspondence that /etc/nginx/conf.d/crowdsec_nginx.conf is considered by Crowdsec to be part of the application distribution and not a user-configurable config file. If this is the case it should be located in /usr/share and symlinked into /etc/ as the current behaviour is contary to established debian packaging conventions. If it is meant to be user-configurable, it should be listed as a Conffile in the package.

This packaging issue was discussed in #93 (comment) but we still don't have a way to configure the cache size that is persistent across upgrades.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions