DocumentJob
Parse a file with a document loader and store the result.
The DataDocument row is created beforehand by POST /api/v1/documents
and already holds the loader name and its validated parameters. The job
copies the source file to <DOCUMENTS_PATH>/<uuid>/source/, parses it,
saves the document under <uuid>/document/ and fills the row.
The source is either an upload or a file inside a completed hub download. Each has its own unit to store it, and both publish the same keys, so the parsing and saving units are shared.
Parameters
kwargs : Dict[str, Any]
document_id (required) and name (optional, for the job list).
Uploads carry file_path, temp_dir and filename, which
POST /api/v1/job/ adds for multipart requests. Hub imports carry
datafile_id and optionally selected_file, the file inside the
download relative to its root.
Methods
get_job_name(self) -> str
DocumentJobGet a descriptive name for the job.
Returns
- str
"Document: <name>"when a name was passed, else"Document load".
on_cancel(self, session_factory: 'sessionmaker' = <function DocumentJob.<lambda> at 0x7fe487a713a0>) -> None
DocumentJobRemove what a killed job left behind and mark the row as error.
run(self) -> None
DocumentJobStore the source file, parse it and fill the document row.
set_status_as_delivered(self, session_factory: 'sessionmaker' = <function DocumentJob.<lambda> at 0x7fe4879d7d80>) -> None
DocumentJobSet the status of the document as delivered.
set_status_as_error(self, session_factory: 'sessionmaker' = <function DocumentJob.<lambda> at 0x7fe487a70f40>) -> None
DocumentJobSet the status of the document as error.
report_progress(self, fraction: Optional[float], message: Optional[str] = None) -> None
BaseJobReport the job's progress to the job queue.
Parameters
- fraction: Optional float
- Completion in the range 0-1, or None when the total work is unknown (the frontend then shows an indeterminate bar).
- message: Optional str
- Short description of the current phase.