Skip to content

save_mvframe returns HTTP 500 for all saves after a session's unlabeled queue is fully drained (JSONDecodeError on blank .unlabeled.jsonl) #95

Description

@walkerjd

Thanks for Lightning Pose and the labeler app! We've just started using it to label
marmoset pose data and was very smooth onboarding so far. Wanted to flag a possible bug,
along with what looks like the root cause and a small fix, in case it's helpful.

Environment

  • lightning-pose 2.2.0, lightning-pose-app 2.2.0.0
  • Python 3.12, running via litpose run_app (Lightning Studio, cloud free tier)

Summary

Once every extracted frame in a session has been labeled (the unlabeled queue
reaches 0), every subsequent POST /app/v0/rpc/save_mvframe returns HTTP 500,
which blocks all further saving for that project until the sidecar file is
manually repaired. Labels do still get written to the CSV (the error happens
after the commit), but the UI shows a generic "An error has occurred /
Error: [object Object]" on every Save, so it isn't obvious what's wrong.

Steps to reproduce

  1. Create a project and extract N frames into a session.
  2. Label and Save all N frames, draining Remaining to 0.
  3. Open any already-labeled frame, change a keypoint, and click Save.
  4. → HTTP 500. Every subsequent save also 500s.

What seems to be happening

Tracing it to litpose_app/routes/labeler/save_mvframe.py, function
remove_from_unlabeled_sidecar_files():

  • When the last entry is removed, lines == [], so this writes the sidecar as a
    single blank line:
    temp_file_path.write_text("\n".join(json.dumps(line) for line in lines) + "\n")
  • On the next save, that blank line is read back through json.loads:
    lines = [json.loads(line) for line in unlabeled_sidecar_file.read_text().splitlines()]
    "\n".splitlines() returns [''], and json.loads('') raises
    JSONDecodeError, which surfaces as the 500.

Possible fix

Skipping blank lines on read resolves it for us (one line):

lines = [json.loads(line) for line in unlabeled_sidecar_file.read_text().splitlines() if line.strip()]

Could optionally also have the writer avoid the trailing blank line for an empty
list. Happy to open a PR if that would be useful.

Workaround (for anyone hitting this)

Truncate the sidecar to 0 bytes: : > <project>.unlabeled.jsonl
— a truly empty file yields [] from splitlines() rather than [''].

Possibly related

Looks like the same queue-bookkeeping area as #51 (closed) and #89 (open),
though this seems to be a distinct failure that only triggers once the queue is
fully drained.

Thanks again for the tool! Happy to test a fix or provide any more detail.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions