Package fr.ens.biologie.genomique.eoulsan.modules.mgmt.hadoop

Class DistCp

java.lang.Object

fr.ens.biologie.genomique.eoulsan.modules.mgmt.hadoop.DistCp

All Implemented Interfaces:

org.apache.hadoop.conf.Configurable, org.apache.hadoop.util.Tool

public class DistCp
extends Object
implements org.apache.hadoop.util.Tool

A Map-reduce program to recursively copy directories between different file-systems.

Since:

1.0

Author:

The hadoop project

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	DistCp.DuplicationException	An exception class for duplicated source files.

Constructor Summary

Constructors

Constructor	Description
DistCp(org.apache.hadoop.conf.Configuration conf)

Method Summary

Modifier and Type	Method	Description
static void	copy(org.apache.hadoop.conf.Configuration conf, String srcPath, String destPath, org.apache.hadoop.fs.Path logPath, boolean srcAsList, boolean ignoreReadFailures)	Deprecated.
org.apache.hadoop.conf.Configuration	getConf()
static String	getRandomId()
int	run(String[] args)	This is the main driver for recursively copying directories across file systems.
void	runWithException(String[] args)	This is the main driver for recursively copying directories across file systems.
void	setConf(org.apache.hadoop.conf.Configuration conf)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

DistCp

public DistCp(org.apache.hadoop.conf.Configuration conf)

Method Detail

setConf

public void setConf(org.apache.hadoop.conf.Configuration conf)

Specified by:

setConf in interface org.apache.hadoop.conf.Configurable

getConf

public org.apache.hadoop.conf.Configuration getConf()

Specified by:

getConf in interface org.apache.hadoop.conf.Configurable

copy

@Deprecated
public static void copy(org.apache.hadoop.conf.Configuration conf,
                        String srcPath,
                        String destPath,
                        org.apache.hadoop.fs.Path logPath,
                        boolean srcAsList,
                        boolean ignoreReadFailures)
                 throws IOException

Deprecated.

Throws:

IOException

run

public int run(String[] args)

This is the main driver for recursively copying directories across file systems. It takes at least two cmdline parameters. A source URL and a destination URL. It then essentially does an "ls -lR" on the source URL, and writes the output in a round-robin manner to all the map input files. The mapper actually copies the files allotted to it. The reduce is empty.

Specified by:

run in interface org.apache.hadoop.util.Tool

runWithException

public void runWithException(String[] args)
                      throws EoulsanException

This is the main driver for recursively copying directories across file systems. It takes at least two cmdline parameters. A source URL and a destination URL. It then essentially does an "ls -lR" on the source URL, and writes the output in a round-robin manner to all the map input files. The mapper actually copies the files allotted to it. The reduce is empty.

Throws:

EoulsanException - if an error occurs

getRandomId

public static String getRandomId()