U d*@sPddlZddlmZddlmZddlmZddlmZm Z GdddeZ dS)N)core) DBFileReader)pipe)Cluster TaskGroupcs<eZdZdZdfdd Zdd Zdd d Zd d ZZS) CachedReaderZ cached_readerLevelDBNdFcs2|dk std||_tt||||||dS)Nzoriginal_reader can't be None)AssertionErrororiginal_readersuperr__init__)selfr db_pathdb_typenameZ batch_sizeZ loop_over __class__?/tmp/pip-unpacked-wheel-ua33x9lu/caffe2/python/cached_reader.pyr >s  zCachedReader.__init__cOs|jjS)zPrepare the reader schema. Since an original reader is given, use it's schema as ground truth. Returns: schema: schema.Struct. Used in Reader.__init__(...). )r Z_schema)rargskwargsrrr_init_reader_schemaRs z CachedReader._init_reader_schemac Cstj|jr|stdgStd}||tRt |j :t (}t |j |jdd|}W5QRXW5QRXW5QRXtd}||td|||gS)aBuild a step for generating cache DB file. If self.db_path exists and not overwritting, build an empty step. Overwise, build a step as follows. Pipe original reader to the _DatasetWriter, so that dataset field blobs are populated. Then save these blobs into a file. Args: overwrite: bool. If true, ignore the existing file and build a new one overwritting the existing one anyway. Returns: build_cache_step: ExecutionStep. The step to be run for building a cache DB file. Z build_stepinit)Z num_threadssaveZ build_cache)ospathexistsrrZexecution_stepZNetZ_init_field_blobs_as_emptyrZ NameScoperrrr dswriterZto_taskZget_step_save_field_blobs_to_db_file)r overwriteZinit_netZcopy_tgZ copy_stepZsave_netrrrbuild_cache_step]s   *  zCachedReader.build_cache_stepcCs*|j|jg|j|j|jdddS)z0Save dataset field blobs to a DB file at db_pathT)dbrZblob_name_overridesZ absolute_pathN)ZSaverZ get_blobsrr field_names)rnetrrrr!|sz)CachedReader._save_field_blobs_to_db_file)rNr F)F) __name__ __module__ __qualname__Zdefault_name_suffixr rr#r! __classcell__rrrrrs0 r) rZ caffe2.pythonrZcaffe2.python.db_file_readerrZcaffe2.python.pipelinerZcaffe2.python.taskrrrrrrrs