CLI Reference¶
The command-line interface (CLI) tool pfs-targetdb-cli is provided to work with the database.
Configuration file
Some commands of the CLI tool introduced below requires a configuration file in TOML format to connect the database.
The configuration file should look like teh following and provided as an option by --config or -c.
dbconf.toml
[targetdb.db]
host = "localhost" # database host
port = 5432 # database port
dbname = "targetdb" # database name
user = "admin" # database user
password = "admin" # database password
dialect = "postgresql" # database dialect
# Optional section for SchemaCrawler
[schemacrawler]
# "_schemacrawler/bin/schemacrawler.sh" under the path will be used
SCHEMACRAWLERDIR = "<path to the schemacrawler package>"
# The following parameters for the uploader will be used to rsync as follows.
# $ rsync -avz -e ssh user@host:data_dir/????/??/????????-??????-{upload_id}
# user can be omitted or blank ("") if the user name is the same as the local user name or an alias is defined in ~/.ssh/config.
[uploader]
host = "<hostname of uploader>"
user = "<user name of uploader>"
data_dir = "<path to the data directory on the uploader>"
# Optional section for Web API access
# The following parameters are used to download data via Web API instead of rsync.
[webapi]
url = "<base URL of the Web API endpoint>" # e.g., "https://example.com/get-upload/"
api_key = "<API key for authentication>" # Optional: leave empty ("") for no authentication
verify_ssl = true # Optional: set to false to disable SSL certificate verification
The schemacrawler section is required only if you want to draw an ER diagram of the database schema with SchemaCrawler.
The webapi section is used by the transfer-targets-api command to download data via Web API.
pfs-targetdb-cli¶
PFS Target Database CLI Tool
Usage:
Options:
--help: Show this message and exit.
Commands:
create-db: Create a database on a PostgreSQL server.drop-db: Drop a database on a PostgreSQL server.install-q3c: Insert the Q3C extension.create-schema: Create tables of the PFS tartedb in a...checkdups: Check for duplicates in data files in a...prep-fluxstd: Prepare flux standard data for the target...diagram: Generate an ER diagram of a database.mdtable: Generate a Markdown output of the schema...insert: Insert rows into a table in the PFS Target...update: Update rows in a table in the PFS Target...parse-alloc: Parse an Excel file containing time...transfer-targets: Download target lists from the uploader to...transfer-targets-api: Download target lists from the uploader to...insert-targets: Insert targets using a list of input...insert-pointings: Insert user-defined pointings using a list...update-catalog-active: Update active flag in the input_catalog...
create-db¶
Create a database on a PostgreSQL server.
Usage:
Options:
-c, --config TEXT: Database configuration file in the TOML format. [required]--help: Show this message and exit.
drop-db¶
Drop a database on a PostgreSQL server.
Usage:
Options:
-c, --config TEXT: Database configuration file in the TOML format. [required]--help: Show this message and exit.
install-q3c¶
Insert the Q3C extension.
Usage:
Options:
-c, --config TEXT: Database configuration file in the TOML format. [required]--help: Show this message and exit.
create-schema¶
Create tables of the PFS tartedb in a database.
Usage:
Options:
-c, --config TEXT: Database configuration file in the TOML format. [required]--drop-all: Flag to drop all tables before creating schema.--help: Show this message and exit.
checkdups¶
Check for duplicates in data files in a directory.
Usage:
Arguments:
DIRECTORY: Directory path containing input files. [required]
Options:
-o, --outdir TEXT: Directory path to save output files. [default: .]--skip-save-merged: Do not save the merged DataFrame.--additional-columns TEXT: Additional columns to output for the merged file. (e.g., 'psf_mag_g' 'psf_mag_r'). The following columns are saved by default: "obj_id", "ra", "dec", "input_catalog_id", "version", "input_file", "is_fstar_gaia", "prob_f_star".--check-columns TEXT: Columns used to check for duplicates. [default: obj_id, input_catalog_id, version]--format [feather|parquet]: File format of the merged data file. [default: parquet]--help: Show this message and exit.
prep-fluxstd¶
Prepare flux standard data for the target database by supplementing additional required fields.
Usage:
Arguments:
INPUT_DIR: Directory path containing input files. Files must be in one of the following formats: parquet, feather, or csv. The input files must be generated in a certain format to be compatible for targetdb. [required]OUTPUT_DIR: Directory path to save the output files. [required]
Options:
--version TEXT: Version string for the F-star candidate catalog (e.g., '3.3'). [required]--input_catalog_id INTEGER: Input catalog ID for the flux standard star catalog.--input_catalog_name TEXT: Input catalog name for the flux standard star catalog.--rename-cols TEXT: Dictionary to rename columns (e.g., '{"fstar_gaia": "is_fstar_gaia"}').--format [feather|parquet]: File format of the output data file. [default: parquet]--help: Show this message and exit.
diagram¶
Generate an ER diagram of a database.
Usage:
Options:
-c, --config TEXT: Database configuration file in the TOML format. [required]--generator [schemacrawler|tbls]: Program to generate ER diagram. [default: schemacrawler]--output-dir TEXT: Directory path to save output files. [default: diagram]--title TEXT: Title of the ER diagram. [default: PFS Target Database]--sc-info-level TEXT: SchemaCrawler info level. [default: maximum]--sc-level-level TEXT: SchemaCrawler log level. [default: SEVERE]--sc-outprefix TEXT: Output file prefix. [default: erdiagram_targetdb]--tbls-format TEXT: tbls format for ER diagrams. [default: mermaid]--help: Show this message and exit.
mdtable¶
Generate a Markdown output of the schema of the PFS Target Database.
Usage:
Options:
-o, --output-file TEXT: Output file.--help: Show this message and exit.
insert¶
Insert rows into a table in the PFS Target Database.
Usage:
Arguments:
INPUT_FILE: Input file to be inserted to targetdb (CSV, ECSV, Feather, or Parquet format). [required]
Options:
-c, --config TEXT: Database configuration file in the TOML format. [required]-t, --table [filter_name|fluxstd|input_catalog|partner|pfs_arm|proposal|proposal_category|sky|target|target_type|user_pointing]: Table name to insert rows. [required]--commit: Commit changes to the database.--fetch: Fetch data from database a the end.--from-uploader: Flag to indicate the data is coming from the PFS Target Uploader. Only required for thetargettable.--flux-type [total|psf]: Flux type for the flux standard star catalog. [default: total]--upload_id TEXT: Upload ID issued by the PFS Target Uploader. Only required for thetargettable.--proposal_id TEXT: Proposal ID (e.g., S24B-QT001). Only required for thetargettable.-v, --verbose: Verbose output.--help: Show this message and exit.
update¶
Update rows in a table in the PFS Target Database.
Usage:
Arguments:
INPUT_FILE: Input file containing data to update records in the PFS Target Database (CSV, ECSV, or Feather formats). [required]
Options:
-c, --config TEXT: Database configuration file in the TOML format. [required]-t, --table [filter_name|fluxstd|input_catalog|partner|pfs_arm|proposal|proposal_category|sky|target|target_type|user_pointing]: Table name to update rows. [required]--commit: Commit changes to the database.--fetch: Fetch data from database a the end.--from-uploader: Flag to indicate the data is coming from the PFS Target Uploader. Only required for thetargettable.--upload_id TEXT: Upload ID issued by the PFS Target Uploader. Only required for thetargettable--proposal_id TEXT: Proposal ID (e.g., S24B-QT001). Only required for thetargettable--verbose: Verbose output.--help: Show this message and exit.
parse-alloc¶
Parse an Excel file containing time allocation information.
Usage:
Arguments:
INPUT_FILE: Path to the Excel file containing time allocation information (e.g., "allocations.xlsx"). [required]
Options:
--output-dir PATH: Directory path to save output files. [default: .]--outfile-prefix TEXT: Prefix to the output files.--help: Show this message and exit.
transfer-targets¶
Download target lists from the uploader to the local machine.
Usage:
Arguments:
INPUT_FILE: Input catalog list file (csv). [required]
Options:
-c, --config TEXT: Database configuration file in the TOML format. [required]--local-dir PATH: Path to the data directory in the local machine [default: .]--force / --no-force: Force download. [default: no-force]--help: Show this message and exit.
transfer-targets-api¶
Download target lists from the uploader to the local machine via Web API.
Usage:
Arguments:
INPUT_FILE: Input catalog list file (csv). [required]
Options:
-c, --config TEXT: Database configuration file in the TOML format. [required]--local-dir PATH: Path to the data directory in the local machine [default: .]--force / --no-force: Force download. [default: no-force]--help: Show this message and exit.
insert-targets¶
Insert targets using a list of input catalogs and upload IDs.
Usage:
Arguments:
INPUT_CATALOGS: Input catalog list to insert (csv). [required]
Options:
-c, --config TEXT: Database configuration file in the TOML format. [required]--data-dir PATH: Path to the data directory. [default: .]--flux-type [total|psf]: Flux type for the flux standard star catalog. [default: total]--commit: Commit changes to the database.--fetch: Fetch data from database a the end.-v, --verbose: Verbose output.--help: Show this message and exit.
insert-pointings¶
Insert user-defined pointings using a list of input catalogs and upload IDs.
Usage:
Arguments:
INPUT_CATALOGS: Input catalog list to insert (csv). [required]
Options:
-c, --config TEXT: Database configuration file in the TOML format. [required]--data-dir PATH: Path to the data directory. [default: .]--commit: Commit changes to the database.--fetch: Fetch data from database a the end.-v, --verbose: Verbose output.--help: Show this message and exit.
update-catalog-active¶
Update active flag in the input_catalog table.
Usage:
Arguments:
INPUT_CATALOG_ID: Input catalog ID to be updated. [required]ACTIVE_FLAG: Active flag to be set. [required]
Options:
-c, --config TEXT: Database configuration file in the TOML format. [required]--commit: Commit changes to the database.-v, --verbose: Verbose output.--help: Show this message and exit.