HDFS Adapter commands overview

The HDFS Adapter can be used to read and write files in HFDS. The adapter uses WebHDFS API to access HDFS. This is a REST API and uses HTTP protocol.

Note: The following is a high level overview of the HDFS Adapter commands. See HDFS Adapter commands for a detailed description of the adapter commands.
  • -S, -H and -P commands - These commands specify the protocol scheme, the host, and the port to which to connect, respectively. The -S command is optional, and it defaults to http, which is the only supported value. Both the -H and the -P commands are required (no default).
  • -U command - Specifies the name of the user known to HDFS.
  • -F command - Specifies the path to the file in HDFS to read or write.
  • -CS - command - Specifies the size (in bytes) of chunks of data the adapter reads from or writes to the file at a time.
  • -SCM command -Enables special mode in a read or write operation where the entire file is read from or written to in a single chunk.
  • -IWM command - Enables another special mode of write operation where the data is written to the target file immediately as it becomes available, instead of being buffered and written when the buffered chunks reach the specified size.
  • -BLS command - Specifies the block size to use for the target file.
  • -BUS command - Specifies the buffer size (in bytes) to use at the transport layer.
  • -OW command - Used to allow overwriting the target file, if the file exists.
  • -CSRFH command -Used to specify the header to set when Cross-Site Request Forgery prevention is enabled in HDFS.
  • -XATTR command - Used to set extended attributes on the target file.
  • -PER command - Specifies the permissions to set on the target file.
  • -REP command - Specifies the replication factor for the target file.
  • -TSL, -TSP, -KSL, -KSP, -KP, -AS, -AC and -VH commands - Used to configure SSL.
  • -KM, -LCFL, -KCFL, -KAU, -KAP and -KCN adapter commands - Used to configure Kerberos.
  • -BAU and -BAP adapter commands - Used for basic HTTP authentication.
  • -T command - This command, and its variants, is used to manage adapter tracing.