$ pip install awscli
For more details such as upgrade, check the amazon webpage here.
Note: if installation was not successful, reinstall forcefully. Check here
The next step is to configure. AWS requires the following items for this:
(how to find these?)
- Access key ID
- Secret Access key
- Region name
- Valid output format
Now, run the following command and answer the prompts:
$ aws configure
AWS Access Key ID [None]: <enter the access key> AWS Secret Access Key [None]: <enter the secret access key> Default region name [None]: <enter region> Default output format [None]: <text>
3. Copying files from AWS S3 folder to Local Desktop
once this is done, we can start using AWS CLI for the file operations. For example, if it want to copy WBB3097.gz file in my AWS account to my local desktop, which is under "qgen" username and in the folder MIX7341, I do the following
$ aws s3 cp s3:URI <Local Desktop Location> $ aws s3 cp s3://qgen/MIX7341/WBB3097.gz ~/Desktop/AWS
4. List all files in AWS S3 folder
Similarly to list all the files in the MIX7341, we can use the following command
$ aws s3 ls s3://qgen/MIX7341/ 2018-08-02 11:03:58 459407360 WBB3066.gz 2018-08-02 11:03:01 337848320 WBB3067.gz 2018-08-02 11:02:57 327219200 WBB3068.gz 2018-08-02 11:02:57 298680320 WBB3069.gz 2018-08-02 11:02:57 280514560 WBB3070.gz
5. Copy multiple files using BASH for loop
Off hand, I had to download multiple files from AWS account. To click each file and download was taking time. So, I just listed the files (step1 below) and used "for loop" in bash to download the files (step2 below).
$ aws s3 ls s3://qgen/MIX7341/ | awk '{print $NF}' >files_in_AWS.list ##step 1 $ for d in $(cat files_in_AWS.list); do echo $d; aws s3 cp s3://qgen/MIX7341/$d . ; done ##step 2
border:solid green;border-width:.1em .1em .1em .8em;padding:.2em .6em;
Python ; Pastie ; www. hilitie.me
