Home

What is MOSAIC?

MOSAIC(Motor-Oral, Speech Analysis, and Integrated Computation) is an articulatory phonetics program, designed to correlate articulatory movement with a Praat analysis Pareselmouth. MOSAIC has quite a lot of features for analysing articulation, and this wiki will cover every feature that MOSAIC has, how to use those features, and how to interpret them.

MOSAIC Structure

MOSAIC is developed in a module system, with the following making up MOSAIC as of 0.3.5-alpha. Please note, modules indicated with an asterisk are in development

• UMD - The Universal Measurement Data (UMD) Structure uses a network of yaml-based drivers to convert varying methods in facial tracking and articulatory analysis into the same columnar structure. It utilises Polars for data compression and Apache Parquet for lightening fast reading and writing. (*)

• Data Wrangling - MOSAIC can automatically correct for head pose (when pose are provided), propagate uncertainty, centre coordinates, and define coordinate anchors.

• Core Measurements - Core measurements are calculated per frame and include the following: coordinate distances (from centre or other coordinates), coordinate angles (theta & phi, from centre or other coordinates), cubic bezier curve coefficients through singular value decomposition, curve area. All core measurement modules are coordinate blind and are controlled using a driver or CLI input.

• Complex Measurements - Complex measurements extrapolate core measurements through the dimension of time. An example is calculating the rate of separation between two coordinates. It calculates first, second, and third derivatives of coordinates (and angles), points (t) along a curve, and area.

• Data Scaling- MOSAIC uses SVD to interpolate articulatory data for segmentation purposes. Interpolated data carries an inherent increase of uncertainty and can not crated artefact's which the initial data lacked (i.e. disfluency which the initial tracking method did not capture).

• Praat Analysis - MOSAIC uses Parselmouth, a python API to use the Praat software for speech analysis.