*iLdZddlmZddlZddlZddlZddlmZddlmZddl m Z ddl m Z m Z mZmZmZ ddlZdZejZejZn#e$r dZd ZdZdZYnwxYwdd lmZdd lmZdd lmZd Z dZ!dZ"dZ#GddZ$ddZ%dS)aMicrophone-based subtask narration aligned with event markers. Pattern (mirrors the validated `xarm_audio_pedal/record_audio.py`): - First audio-pedal press during an active recording starts a continuous PyAudio input stream. - Each subsequent press marks a segment boundary in the running stream. - On episode stop, the stream is closed and one WAV is written per segment under ``/audio/`` along with a JSON sidecar. Boundary timestamps share the camera/robot clock used by ``event_markers`` (via the ``capture_clock`` callable injected by ``DemonstrationRecorder``), so audio narration aligns with `robot_data.npz` and converted camera frames without a post-hoc shim. ) annotationsN)deque)datetime)Path)DequeDictListOptionalTupleTF) CLOCK_CAMERA)CLOCK_FALLBACK)warniiceZdZdZd$d%dZd&dZd&d Zd'd Zd(dZd&dZ dZ d)d*dZ d+dZ e d,dZd&dZd&dZd-d#ZdS). AudioRecorderuContinuous microphone recorder driven by ``mark_boundary`` calls. One instance per session. Episode-scoped state (segments, frames, recording_dir) is rebuilt at the start of each episode. Boundaries are pushed in by the recorder thread via ``mark_boundary`` (which carries the same ``(t_ns, clock)`` value used for the matching ``event_markers`` entry). This guarantees that ``event_markers[i].t`` and ``audio_segments[i].boundary_t_ns`` are bit-identical — there is no second clock read on the audio thread. Lifecycle:: ar = AudioRecorder() ar.start_session() # spins the daemon thread ar.start_episode(rec_dir) # at the top of each episode ar.mark_boundary(ts_ns, clock) # called by the recorder loop on each press ar.stop_episode() # signals end of episode ar.wait_until_idle() # block until WAVs are flushed audio = ar.drain() # {"audio_full": ..., "audio_segments": [...]} ar.stop_session() # at session end The recorder is fail-soft: if PyAudio is missing or the input device can't be opened, ``start_session`` prints a yellow warning and the recorder becomes a no-op (boundaries are ignored, no WAVs written, the rest of recording continues). Callers don't need to special-case this. N device_index Optional[int]returnNonec.||_d|_d|_d|_d|_d|_t j|_t j|_ |j g|_ d|_ t|_dS)NF) _device_index_pa_enabled_session_running_thread _episode_dir threadingEvent_episode_running_idleset _segments_fullr_pending_boundaries)selfrs */home/robot-lab/raiden_cmu/raiden/audio.py__init__zAudioRecorder.__init__Ss)04 !&37 -1 ) 1 1_&&  %'%) , "audio_segments": [...]}``. ``audio_full`` is the single concatenated WAV covering the whole episode (always written when at least one frame was captured). ``audio_segments`` is one entry per pedal press — empty when the operator never pressed during this episode. ) audio_fullaudio_segmentsN)r$listr#)r&outs r'drainzAudioRecorder.drains7*"4>22    r)c|jS)zGTrue if the recorder will actually capture audio (PyAudio + device OK).)rrCs r'enabledzAudioRecorder.enableds }r)c |jrI|jstjd5||jInH#t ttf$r.}tdt|j d|dd}~wwxYw|j dS#|j wxYw)N皙?z ! AudioRecorder thread died (: ))rr rMtimesleep_capture_one_episoder4 RuntimeError ValueErrorr:type__name__r!r"r;s r'r8zAudioRecorder._runs ' ,,3355Jt$$$))+++ ' , z2    LDGG4DLLLLL M M M    JNN     DJNN    s*AAB6B*)BBB66Cc |j}||jdSg}g}|fd} |jt t td|jt|}nK#ttf$r7}td|d|jYd}~dSd}~wwxYw|j td |jr|jr|jr|j\}}|t'|||t)jftdt'|d |d ||jt)jd |jr|j ||n&#tt0f$rtd YnwxYw|jrZ|j\}}|t'|||t)jf|jZn# ||n&#tt0f$rtd YnwxYw|jrZ|j\}}|t'|||t)jf|jZwxYw |||||j dS#|j wxYw) urOpen the stream at episode start, capture continuously, save on stop. Audio is captured from the moment ``start_episode`` is called until ``stop_episode`` clears ``_episode_running``, but **only the audio from the first press onwards is written to disk** — the pre-first-press period is treated as warm-up noise and discarded. Boundaries are drained from ``_pending_boundaries`` (pushed by the recorder thread via ``mark_boundary``), producing one WAV per inter-press interval plus a single ``audio_full.wav`` covering first-press → end-of-episode. Nc>||dtfSN)rN_PYAUDIO_PA_CONTINUE)in_data frame_count time_infostatus_fs r' _callbackz5AudioRecorder._capture_one_episode.._callbacks IIg   ./ /r)T)formatchannelsrateinputinput_device_indexframes_per_bufferstream_callbackz! ! AudioRecorder.open() failed (z); audio off for this episodeuC ✓ Audio stream open (warm-up noise discarded until first press)u ✓ Audio segment boundary #z @ ts=z clock=rbz3 ! AudioRecorder stream cleanup raised; continuing)rr rAropen_PYAUDIO_FORMAT_INT16CHANNELS SAMPLE_RATErCHUNKr4r5r:r!rrMr%popleftrNlenrerf stop_streamcloserh _save_audior") r& episode_dirframes boundariesrustreamr<ts_nsrJs r'rgz"AudioRecorder._capture_one_episodes'    ! ' ' ) ) ) F 8: CI 0 0 0 0 X]],! #'#5"' )#FF!    VaVVV W W W  ! ' ' ) ) ) FFFFF   STTT L' !D,A,H,H,J,J ! .#'#;#C#C#E#ELE5%%s6{{E5$)++&NOOO6Z66 %66.366. 4   ' !D,A,H,H,J,J ! M""$$$ \* M M MKLLLLL M * L#7??AA u!!3v;;udikk"JKKK* L M""$$$ \* M M MKLLLLL M * L#7??AA u!!3v;;udikk"JKKK* L L L L L    [&* = = = JNN     DJNN    sg:A**B2;,B--B2CI*9(G"" HH*L,(JL J85L7J88A%L!MM.rr List[bytes]r!List[Tuple[int, int, str, float]]c L|sdS|dz }|ddtjd}|d\}}}} t j} |t |d|| fgz} t t | dz D]} | | \} }}}| | dz\}}}}|| |}d| d |d }t||z |}|| |||t|d tt|d }|d| d |d z tj |d|j|| |t|d |dt!d| d|d|dd|d d}t||z ||d}||| t|d tt|d}|dz tj |d||t|d |d|_t!d|d|dd|ddS)uIWrite per-press segment WAVs + a concatenated ``audio_full.wav``. Both shapes start at the **first pedal press** — pre-first-press frames are warm-up noise and discarded. With N presses you get N segments (segment ``i`` runs from press ``i`` to press ``i+1``, last → end-of-episode) plus one ``audio_full.wav`` that is sample-identical to the concatenation of those segments. Nothing is written if the operator never pressed the pedal. We gate only on ``boundaries`` (not ``frames``): a press recorded before the first PyAudio callback delivered a buffer would otherwise drop the boundary entirely, breaking the ``len(audio_segments) == len(event_markers)`` contract. Empty slices yield valid header-only WAVs with ``duration_s = 0.0``. NaudioT)parentsexist_okz %Y%m%d_%H%M%Srraudio__z.wav) audio_file segment_id boundary_t_nsboundary_wall_time end_wall_time duration_s sample_raterwrJz.jsonr)indent)rrrrrJu ✓ Saved audio segment rcz (z.1fz s, clock=rdzaudio_full.wav)r start_t_nsstart_wall_timerrrwrJzaudio_full.json)rrrrJu ✓ Saved audio_full: )mkdirrnowstrftimererrange _save_wavroundrr write_textjsondumpsr#rNr:r$)r&rrr audio_dir timestamp_str first_idx first_ts_ns first_clock first_wallend_wallboundaries_with_endi start_idx start_ts_ns start_clock start_wallend_idxr end_wall_i seg_frames wav_filenamedurationsidecar full_wav_name full_duration full_sidecars r'rzAudioRecorder._save_audio^s*  F') t444 //@@ :DQ-7 ; Z9;;( [[![( 3,  s.//!344( ( A>QRS>T ;I{K(;AE(B %GQ: ' 12Jrs  #"""""  55555555555555  NNN#O"-   G   877777;;;;;;&&&&&&   L L L L L L L L h - - - - - -sA AA