*iBdZddlZddlZddlZddlZddlmZddlmZddl Z ddl Z d!de j de de j fd Zde j d ed ed ed ede j de j fdZd"dedeeeffdZ d#dedede dededede ddfd ZdS)$z;Visualize a converted UnifiedDataset recording using Rerun.N)Path)Tuple_depth_m percentilereturnc tj|dkdtj|ddz d}||dk}t|dkr"t tj||nd}||dzz dd}t j|dztj t j }t j |t j }d||dk<|S)uAPlasma-colorize a depth map using inverse-depth mapping. Mirrors ``viz_depth`` / ``viz_inv_depth`` from AnyData: - compute inverse depth (1 / depth), masking zeros - normalize by the *percentile*-th value of valid pixels - apply plasma colormap; invalid pixels stay black Returns uint8 H×W×3. r?g:0yE>Ngư>) npwherecliplenfloatrcv2 applyColorMapastypeuint8COLORMAP_PLASMAcvtColor COLOR_BGR2RGB)rrinvvalid normalizerinv_norm colored_bgr colored_rgbs //home/robot-lab/raiden_cmu/raiden/visualizer.py _viz_depthr s (7Q;bggtT&B&B BC H HC aLEK,{C,=>>KKq fxfycxcy T_cam2worldc|j\}}tjtj|tjtj|tj\}} |d} tj|d|z | z|z | d|z | z|z | tj| gd} || jzjddddf} | S)zUnproject every pixel into world space. Returns all (H*W, 3) world-space points (including invalid ones so the caller can apply a uniform validity mask that aligns with flattened images). dtypeaxisN) shaper meshgridarangefloat32reshapestack ones_likeT) rr"r#r$r%r&HWuuvvdpts_cam pts_worlds r_reconstruct_pointsr>)s =DAq [1BJ7771BJ9W9W9W X XFBAh ZZ^^b A % * ZZ^^b A % * LOO  Gwy(+AAArrE2I r!data/processeddata_dirct|}td|D}|std|ddlm}d|D}|t |dd}||}td|D}|std |i}|D]} | j} t| d z 5} tj | } d d d n #1swxYwY| d i d d} | r | jd| d} | || <|t |dd}t|||jfS)zTUse fzf to select a task then an episode. Returns ``(task_dir_path, episode_name)``.c3K|]E}|td|DA|VFdS)c3lK|]/}||dz V0dS metadata.jsonNis_direxists).0eps r z4select_task_and_episode...Rs@UUBUo%--//UUUUUUr!N)rGanyiterdirrIr;s rrKz*select_task_and_episode..Nsm 88:: UU!))++UUU U U  r!zNo converted tasks found in r) fzf_selectci|] }|j| S)namerNs r z+select_task_and_episode..Ys000161000r!zVisualize task> )promptc3pK|]1}||dz -|V2dSrDrFrNs rrKz*select_task_and_episode..]s\9:_9L8T8T8V8V r!zNo episodes found in rENinforaw_idz ()zSelect episode> ) rsortedrMFileNotFoundError raiden.utilsrOlistrRopenjsonloadgetstr)r@base task_dirsrO task_labels chosen_tasktask_dir episode_dirsepisode_labelsrJlabelfep_metarW chosen_eps rselect_task_and_episodernKs$ >>DI G Et E EFFF''''''00i000K*T+..7IJJJ1MK;'H##%%L D B B BCCCN## "& ' ' #1illG # # # # # # # # # # # # # # #VR((,,Xr::  -w,,6,,,E "u 4//8JKKKANI x==.38 88s'DD D 0000r+?皙?F# recording_direpisodestride image_scale frustum_scalewebweb_portcOddl}tttjj}|t jddvr=|t j zt jddzt jd<t|} | |z } | std| t| dz 5} tj| } dddn #1swxYwY| dg} | did d }| d id dg}|r|dnd}| d z OOfd| D}|stdOt!|}t#d| jt#d|t#d|t#d| t#d|t'jdt&j}| dz dz }| rst|d5}t-j|}dddn #1swxYwY|d}| t'j|t&j}|rddlm}|d|dz}||}||d d!|d"||d#}t#d$|t#d%|d&|d'|d&|d( t#n|dd)*|d|d+d| jd|d|dd,| d|g|j j!-d).tEtGd||}t#d/tI|d0|d1|}gg}}tG|D]} | dz | d2d3z }!|! rt|!d5}t-j|}"dddn #1swxYwYd4|"vr|"d4}#|%|#dd5t'j|#d6d7t&j}$|t'j%|$d8zdd5}%|%|%&t&j'|rP|d9|(t'j|t&j'ggd:gd;<d).|rP|d=|(t'j|t&j'ggd>gd;<d).tS|D]\}&}'|*d?|'@|'d2}(d})| D]}*dA|*}+| dz |(d3z },|, s(t|,d5}t-j|}-dddn #1swxYwY|-dB|*}.|-dC|*}/|.|/|.&t&j}.|/&t&j}/|.dD|z}0|.dE|z}1|.dF|z}2|.dG|z}3||+|+|/dd5d5f|/dd5dd5fHd}4| d z |*z |(dIz }5|5 rktYj-t|5}6|6G|6j.ddJ\}7}8t_dta|7|z}9t_dta|8|z}:|d8kr#tYj1|6|:|9ftXj2K}6tYj3|6tXj4}4t'j|0d|2gd|1|3ggdLgt&j'}.||+|5|.|:|9|M||+|6|4||+dN|6|4| dOz |*z |(dPz };|; rt'j|;dO}<|<&t&j'dQz }=|4$tYj1|=|:|9ftXj7K}=no|d8kri|=j.\}>}?tYj1|=t_dta|?|zt_dta|>|zftXj7K}=||+dR|6tq|=ts|=|0|1|2|3|/}@t'j:|@;dSt'j<|@=dSdTkz}A|@|A}BtI|BdkrHi}C|4|4>dUd5|A|CdV<|dW|*|j?|Bfi|C|)sd4|-vr|-d4}D|Ddd5}E|Dd5dX>d5d5}Ft|DdX}Gt'j|Dd6d7t&j}H|t'j%|Hd8zdd5&t&j'}I|dd5dd5f|Dd7dY>d5d5z&t&j'}Jt|DdY}KtSgdZD]\}L}M|d[|M|At|I|L|d\|M|At|E|L|d]|A|K|d^|A|G|d_|+|I|JH|d`|+|E|FH|da|?|Iggd:gdb<|dc|?|Eggd>gdb<d)})|&dzddzdks|&tI|dz kr*t#de|&dzdfdgtI|dh|'di|rKt#dj ddlB}N |NCd#t$rt#dkYdSwxYwt#dldS)maVisualize a converted recording directory using Rerun. Mirrors RerunDisplay from AnyData: logs camera frustums (Transform3D + Pinhole), RGB images (at entity path for frustum texture and at ``{entity}/rgb`` for the 2-D view), plasma-colorized depth maps, and colored world-space point clouds. Parameters ---------- recording_dir: Path to the converted task directory (e.g. ``data/processed/pick_purrito``). episode: Episode subdirectory name (default: ``"0000"``). stride: Log every *stride*-th frame (default: ``1``). image_scale: Uniform downsample factor for images and point clouds (default: ``0.25``). frustum_scale: ``image_plane_distance`` passed to ``rr.Pinhole``; controls the rendered frustum size in the 3-D view (default: ``0.1``). web: Serve the viewer over HTTP instead of spawning the native desktop app. Use this when connecting via SSH tunnel. web_port: HTTP port for the web viewer (default: ``9090``). rNPATHrXzEpisode directory not found: rEcamerasrVrRunknownlanguagerTrgbc g|]K}|z tt|z dLS)z*.png)rHrrZglob)rIcamrgb_dirs r z'visualize_recording..sa  cM ! ! # # FGcM''00 1 122r!zNo RGB frames found under z Recording : z Task : z Prompt : z Cameras : z Frames : r(lowdimz0000000000.pklrbT_left_from_right)quoteraidenr+) grpc_portF)ry open_browserzhttp://localhost:z?url=)safez Open in browser: zSSH tunnel: ssh -L z :localhost:z -L z T)spawn z, ) media_type)staticzLogging z frames (stride=z) ...010dz.pklactionr. r ztrajectory/right)P<g~jth?)colorsradiiztrajectory/left)rxrframe)sequencezworld/cameras/ intrinsics extrinsics)rr)r+r+)r)r+r) translationmat3x3z.pngr) interpolation)rrr+)image_from_camerawidthheightimage_plane_distancez/rgbdepthz.npzg@@z/depthr,r r*rz world/points/ )xyzz action/right/z action/left/zaction/right/gripperzaction/left/gripperzworld/ee/rightz world/ee/leftztrajectory/right/currentg{Gz?ztrajectory/left/currentz z>4d/z (frame rYu/Done logging. Serving — press Ctrl-C to stop.z Stopped.z&Done. Rerun viewer should now be open.)Ererunrbrsys executableparentosenvironrapathseprHr[r^r_r` RuntimeErrorminprintrRr eyefloat64picklearray urllib.parserinit serve_grpcserve_web_viewerlog TextDocumentjoin MediaTypeTEXTr]rangerappendrr2 LineStrips3D enumerateset_time Transform3Drimreadr/maxintresize INTER_AREArrPinholeImage INTER_NEARESTr r>isfiniteallabssumr3Points3DrScalarstimesleepKeyboardInterrupt)Prsrtrurvrwrxryrr _venv_binrec_direp_dirrkmeta cam_names task_namepromptsrTcountsn_framesr _first_lowdim_f_ld0_trr server_uri viewer_url frame_lists left_pos_listright_pos_listfipaactr_pos_rbr_pos_lblog_idx frame_idx frame_name action_loggedcam_nameentity lowdim_pathldKT_c2wfx_sfy_scx_scy_simg_rgbrgb_pathimg_bgrH0W0H_dW_d depth_pathdepth_mmrHdWdptsr pts_validkwargsrl_posl_rotl_grip _r_pos_rbr_posr_rotr_gripjlblrrsP @rvisualize_recordingrvs F D((/00I vr2222&3bjnnVR6P6PP 6=!!G w F ==??J H H HIII f& ' '1y||hhy"--IXXfb))--fi@@Ihhz2&&**8bT::G '/'!**RFunGF CAAABBB6{{H ' ' '((( $ $ $%%% ! ! !""" $ $ $%%% # # #$$$%'F1BJ$?$?$?X%(88M? - & & #";r??D # # # # # # # # # # # # # # # XX) * * > "2: > > >  &&&&&&& qL ]]Y]77  XEBBBTTTjr8R8R8RTT  0J00111 rx r rH r r) r r`i r r r     %%%FF  II17<11.9..+6++9499Y#7#799-8--   |(   "eAx0011J CS__ C Cf C C CDDDA%'>MHoo C C X 2 0 0 0 0 0 88:: Ca $"KOO $ $ $ $ $ $ $ $ $ $ $ $ $ $ $1}}k$$S1X...8C2JbjAAA- (C0H0HH"1"M%%hoobj&A&ABBB     OO. ;;;<%              OO-rz:::;&'        ( 33RQRQ Gi 000!((  !J %J %H0h00F !8+.A.A.AAK%%'' k4(( %B[__ % % % % % % % % % % % % % % %< $$X..A|$((22EyEM$$ALL,,ET7Q;DT7Q;DT7Q;DT7Q;D FF5!Q<bqb"1"f NN   G~0j3F3F3FFH   ?*S]]33&$]2A2.FBaR!V--CaR!V--CCxx"%*#c3Zs~###"l7C4EFFG41dD/999ERZAFF ./"%#&1> #FF6288G#4#4555FFf???BHHW,=,=>>>  ')H4*7J7J7JJJ  "" Y7:..w7"//"*55>&!j#s3;LGG#XX$]FB!jQBF ,,c!Sa[[.A.AB&)&7G &((("((:g3F3F*G*GHHH*'4tT5QQ C((,,!,44s Q8O8ORV8VWJ y>>A%%#%F*+2??2q+A+A%+Hx(FF5855{r{97W7WPV7W7WXXX !% %X^^%'\qs qt ,,Q22vbz**HVBrE]"*EEE *RYy#-F-FFKRRJ&bqb"1"f-r"u 0E0Ea0K0KK&$$vbz**(88NNFAsFF0300"**U58__2M2MNNNFF/#//E%(OO1L1LMMMM-rzz&/A/ABBB,bjj.@.@AAA$bnnun&U&U5QV(W(WXXX.KKtKLL-KK0@KMM!% aK2  " "gZ11D&D&D Ow{OOOZOO9OOO P P P 8 ?@@@ KKK  1      ,        677777sZC==DD JJJ8RR R Y::Y> Y> x!!x>=x>)r)r?)ror+rprqFrr)__doc__r_rrrpathlibrtypingrrnumpyr ndarrayrr rr>rbrnboolrrQr!rr$sAA   RZ. Z        ZD#9#9c#9sCx#9#9#9#9ZA8A8A8 A8 A8 A8  A8  A8A8 A8A8A8A8A8A8r!