[cs615asa] [git commit] CS615 EBS-BACKUP; backup a directory into Elastic Block Storage (EBS) branch main updated. ff8187d23fc1ec4bf68cc1a782059fb1caa32515

Git Owner jschauma at stevens.edu
Mon May 10 15:17:03 EDT 2021


This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "CS615 EBS-BACKUP; backup a directory into Elastic Block Storage (EBS)".

The branch, main has been updated
       via  ff8187d23fc1ec4bf68cc1a782059fb1caa32515 (commit)
      from  8921b4e46c664e4dcf60035980e4af77dada5e61 (commit)

Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.

- Log -----------------------------------------------------------------
commit ff8187d23fc1ec4bf68cc1a782059fb1caa32515
Author: Hunter <hdevlin at stevens.edu>
Date:   Mon May 10 15:11:27 2021 -0400

    added docs for main.py

diff --git a/README b/README
index cc665a7..2f21b94 100644
--- a/README
+++ b/README
@@ -20,8 +20,8 @@ automatically to cs615asa at lists.stevens.edu.
 To install the `ebs-backup(1)` tool, you must have pip. To install, run:
 
 ```sh
-pip install setuptools
-pip install -e .
+pip3 install setuptools
+pip3 install -e .
 ```
 
 Then the command `ebs-backup(1)` is usable directly in the command line.
\ No newline at end of file
diff --git a/src/ebs_backup.egg-info/PKG-INFO b/src/ebs_backup.egg-info/PKG-INFO
deleted file mode 100644
index 456b8c7..0000000
--- a/src/ebs_backup.egg-info/PKG-INFO
+++ /dev/null
@@ -1,184 +0,0 @@
-Metadata-Version: 1.2
-Name: ebs-backup
-Version: 0.0.1
-Summary: Tool to backup ebs volumes
-Home-page: UNKNOWN
-Author: 2021S CS615
-Author-email: hdaly1 at stevens.edu
-License: MIT
-Description: EBS-BACKUP(1)		NetBSD General Commands Manual		 EBS-BACKUP(1)
-        
-        NAME
-             ebs-backup -- backup a directory into Elastic Block Storage (EBS)
-        
-        SYNOPSIS
-             ebs-backup [-h] [-l filter] [-r filter] [-v volume-id] dir
-        
-        DESCRIPTION
-             The ebs-backup tool performs a backup of the given directory into Amazon
-             Elastic Block Storage (EBS).  This is achieved by creating a volume of
-             the appropriate size, attaching it to an EC2 instance and finally copying
-             the files from the given directory onto this volume.
-        
-        OPTIONS
-             ebs-backup accepts the following command-line flags:
-        
-             -h		   Print a usage statement and exit.
-        
-             -l filter	   Pass data through the given filter command on the local
-        		   host before copying the data to the remote system.
-        
-             -r filter	   Pass data through the given filter command on the remote
-        		   host before writing the data to the volume.
-        
-             -v volume-id  Use the given volume instead of creating a new one.
-        
-        DETAILS
-             ebs-backup will perform a backup of the given directory to an ESB volume.
-             The backup is done with the help of the tar(1) command on the local host,
-             writing the resulting archive directly to the block device.  That is,
-             ebs-backup does not create or use a filesystem on the volume.  Instead,
-             ebs-backup utilizes the dd(1) command to write out the data to the raw
-             volume.  In essence, ebs-backup wraps the following pipeline:
-        
-        	   tar cf - <dir> [| local-filter] | ssh ec2-instance "[remote-filter |] dd of=/dev/xbd2d"
-        
-             Here, "/dev/xbd2d" stands for the suitable raw disk device, which may
-             differ depending on the instance type.
-        
-             ebs-backup does not use any temporary files, nor creates a local copy of
-             the archive it writes to the volume.
-        
-             ebs-backup can pass the archive it creates through a filter command on
-             either the local or the remote host.  This allows the user to e.g. per-
-             form encryption of the backup.
-        
-             Unless the -v flag is specified, ebs-backup will create a new volume, the
-             size of which will be at least two times the size of the directory to be
-             backed up.
-        
-             ebs-backup will create an instance suitable to perform the backup, attach
-             the volume in question and then back up the data from the given direc-
-             tory.  Afterwards, ebs-backup will terminate the instance it created.
-        
-             ebs-backup will not create or modify any other AWS resources.
-        
-        VERIFICATION
-             By default, ebs-backup simply writes the data to the volume.  To verify
-             that this was successful, the user may manually perform the following
-             tasks:
-        
-        	   aws ec2 run-instances
-        	   aws ec2 attach-volume
-        	   ssh instance "dd if=/dev/xbd2d" | tar tvf -
-        
-        OUTPUT
-             If successful, ebs-backup will print the volume-id of the volume to which
-             it backed up the data as the only output.
-        
-             Unless the EBS_BACKUP_VERBOSE environment variable is set, ebs-backup
-             will not generate any other output unless any errors are encountered.  If
-             that variable is set, it may print out some useful information about what
-             steps it is currently performing.
-        
-             Any errors encountered cause a meaningful error message to be printed to
-             STDERR.
-        
-        ENVIRONMENT
-             ebs-backup assumes that the user has set up their environment for general
-             use with the EC2 tools and ssh(1) without any special flags on the com-
-             mand-line.	 That is, the user has a suitable section in their ~/.ssh/con-
-             fig file to ensure that running 'ssh ec2-instance.amazonaws.com' suc-
-             ceeds.
-        
-             To accomplish this, the user has created an SSH key pair named 'ebs-
-             backup' and configured their SSH setup to use that key to connect to EC2
-             instances.
-        
-             Therefor, ebs-backup will not set nor modify the variables AWS_CON-
-             FIG_FILE, EC2_CERT, EC2_HOME or EC2_PRIVATE_KEY.
-        
-             ebs-backup allows the user to add custom flags to the commands related to
-             starting a new EC2 instance via the EBS_BACKUP_FLAGS_AWS environment
-             variable.
-        
-             ebs-backup also assumes that the user has set up their ~/.ssh/config file
-             to access instances in EC2 via ssh(1) without any additional settings.
-             It does allow the user to add custom flags to the ssh(1) commands it
-             invokes via the EBS_BACKUP_FLAGS_SSH environment variable.
-        
-             As noted above, the EBS_BACKUP_VERBOSE variable may cause ebs-backup to
-             generate informational output as it runs.
-        
-        EXIT STATUS
-             The ebs-backup will exit with a return status of 0 under normal circum-
-             stances.  If an error occurred, ebs-backup will exit with a value >0.
-        
-        EXAMPLES
-             The following examples illustrate common usage of this tool.
-        
-             To back up the entire filesystem:
-        
-        	   $ ebs-backup /
-        	   vol-1a2b3c4d
-        
-             To create a complete backup of the current working directory using
-             defaults to the volume with the ID vol-1a2b3c4d, possibly overwriting any
-             data previously stored there:
-        
-        	   $ ebs-backup -v vol-1a2b3c4d .
-        	   vol-1a2b3c4d
-        
-             To do the same thing again, but having the program tell us what it's
-             doing, the user can set the EC2_BACKUP_VERBOSE environment variable.
-             Possible diagnostic messages generated when that variable is set are
-             shown below:
-        
-        	   $ EBS_BACKUP_VERBOSE=1 ebs-backup -v vol-1a2b3c4d .
-        	   Verifying volume...
-        	   Volume vol-1a2b3c4d is in availability zone 'us-east-1a'.
-        	   Creating a suitable instance in 'us-east-1a'...
-        	   Attaching volume vol-1a2b3c4d to instance i-123abcd456...
-        	   Performing backup...
-        	   Terminating instance i-123abcd456...
-        	   Backup complete, 1.7 GB of data written to:
-        	   vol-1a2b3c4dgg
-        
-             Suppose a user has their ~/.ssh/config set up to use the private key
-             ~/.ec2/stevens but wishes to use the key ~/.ssh/ec2-key instead:
-        
-        	   $ export EBS_BACKUP_FLAGS_SSH="-i ~/.ssh/ec2-key"
-        	   $ ebs-backup .
-        	   vol-1a2b3c4d
-        
-             To force creation of an instance type of t1.micro instead of whatever
-             defaults might apply
-        
-        	   $ export EBS_BACKUP_FLAGS_AWS="--instance-type t1.micro"
-        	   $ ebs-backup .
-        	   vol-1a2b3c4d
-        
-             To locally encrypt the backup of the '/var/secrets' directory:
-        
-        	   $ ebs-backup -l 'gpg -e -r 9BED3DD7' /var/secrets
-        	   vol-1a2b3c4d
-        
-             The same as above, but perform encryption on the remote system:
-        
-        	   $ ebs-backup -r 'gpg -e -r 9BED3DD7' /var/secrets
-        	   vol-1a2b3c4d
-        
-        SEE ALSO
-             aws help, cat(1), dd(1), ssh(1), tar(1)
-        
-        HISTORY
-             ebs-backup was originally assigned by Jan Schaumann
-             <jschauma at cs.stevens.edu> as a homework assignment for the class "Aspects
-             of System Administration" at Stevens Institute of Technology in the
-             Spring of 2011.
-        
-        NetBSD 8.0		       February 7, 2021			    NetBSD 8.0
-Platform: UNKNOWN
-Classifier: License :: OSI Approved :: MIT License
-Classifier: Programming Language :: Python :: 3.7
-Requires-Python: >=3.7
diff --git a/src/ebs_backup.egg-info/SOURCES.txt b/src/ebs_backup.egg-info/SOURCES.txt
deleted file mode 100644
index 715c026..0000000
--- a/src/ebs_backup.egg-info/SOURCES.txt
+++ /dev/null
@@ -1,12 +0,0 @@
-README
-ebs-backup.1
-ebs-backup.1.gz
-setup.py
-src/ebs_backup.egg-info/PKG-INFO
-src/ebs_backup.egg-info/SOURCES.txt
-src/ebs_backup.egg-info/dependency_links.txt
-src/ebs_backup.egg-info/entry_points.txt
-src/ebs_backup.egg-info/requires.txt
-src/ebs_backup.egg-info/top_level.txt
-test/test_arg_parsing.py
-test/test_env_parsing.py
\ No newline at end of file
diff --git a/src/ebs_backup.egg-info/dependency_links.txt b/src/ebs_backup.egg-info/dependency_links.txt
deleted file mode 100644
index 8b13789..0000000
--- a/src/ebs_backup.egg-info/dependency_links.txt
+++ /dev/null
@@ -1 +0,0 @@
-
diff --git a/src/ebs_backup.egg-info/entry_points.txt b/src/ebs_backup.egg-info/entry_points.txt
deleted file mode 100644
index c46fe94..0000000
--- a/src/ebs_backup.egg-info/entry_points.txt
+++ /dev/null
@@ -1,3 +0,0 @@
-[console_scripts]
-ebs-backup = main:main
-
diff --git a/src/ebs_backup.egg-info/requires.txt b/src/ebs_backup.egg-info/requires.txt
deleted file mode 100644
index f0dbda1..0000000
--- a/src/ebs_backup.egg-info/requires.txt
+++ /dev/null
@@ -1,10 +0,0 @@
-boto3
-botocore
-environs
-jmespath
-marshmallow
-python-dateutil
-python-dotenv
-s3transfer
-six
-urllib3
diff --git a/src/ebs_backup.egg-info/top_level.txt b/src/ebs_backup.egg-info/top_level.txt
deleted file mode 100644
index a9a5aec..0000000
--- a/src/ebs_backup.egg-info/top_level.txt
+++ /dev/null
@@ -1 +0,0 @@
-tmp
diff --git a/src/main.py b/src/main.py
index 31910c9..2fc18be 100644
--- a/src/main.py
+++ b/src/main.py
@@ -54,12 +54,26 @@ session = boto3.Session(
 client = boto3.client('ec2')
 
 def calculate_dir_size(dir):
+    """Helper function to find the size of input directory
+
+    Args:
+        dir (string): directory that the size is determined of.
+
+    Returns:
+        int: total size of input directory
+    """
     # TODO: make sure this is x-platform compatible
     total = sum(d.stat().st_size for d in os.scandir(dir) if d.is_file())
     return total
 
 
 def pipe_backup(args, target_ip):
+    """ssh into ec2 instance and moves the argument directory
+
+    Args:
+        args : command line arguments, parsed by argument_parsing.py
+        target_ip : Public dns IP of ec2 instance
+    """
     local_filter = args.l
     remote_filter = args.r
     target_dir = args.dir
@@ -89,6 +103,12 @@ def pipe_backup(args, target_ip):
 
 
 def wait_for_instance_ok(instance_id, tries):
+    """Continually checks status of instance to see when ssh can be run on ec2
+
+    Args:
+        instance_id : id of ec2 instance that the status is being checked of
+        tries (integer): number of times status is checked
+    """
     i = 1
     status_dictionary = client.describe_instance_status(InstanceIds=[instance_id])
     instance_status_info = status_dictionary.get('InstanceStatuses')
@@ -112,6 +132,13 @@ def wait_for_instance_ok(instance_id, tries):
             print('Instance {id} Status: ok'.format(id=instance_id))
 
 def wait_for_volume_status(volume_id, target_status, tries):
+    """continually checks volume status, and compares to target_status
+
+    Args:
+        volume_id: volume id of ec2
+        target_status : desired status of volume
+        tries (integer): number of times volume status is checked
+    """
     i = 1
     status_dictionary = client.describe_volumes(VolumeIds=[volume_id])
     volume_status = status_dictionary.get('Volumes')[0].get('Attachments')[0].get('State')
@@ -138,11 +165,24 @@ def wait_for_volume_status(volume_id, target_status, tries):
 
 ### Always cleanup ###
 def exit(ec2):
+    """exit and cleanup EC2
+    this is called as an atexit, and will always run at termination
+
+    Args:
+        ec2 : active ec2 session to be closed
+    """
     def handle():
         ec2.cleanup()
     return handle
 
 def backup(args):
+    """ 'main' function
+    creates ec2 instance, attaches volume, moves directory by ssh, closes and
+    cleans up the ec2
+
+    Args:
+        args : command line arguments, parsed by argument_parsing.py
+    """
     ec2 = None
     try:
         size = calculate_dir_size(args.dir)

-----------------------------------------------------------------------

Summary of changes:
 README                                       |   4 +-
 src/ebs_backup.egg-info/PKG-INFO             | 184 ---------------------------
 src/ebs_backup.egg-info/SOURCES.txt          |  12 --
 src/ebs_backup.egg-info/dependency_links.txt |   1 -
 src/ebs_backup.egg-info/entry_points.txt     |   3 -
 src/ebs_backup.egg-info/requires.txt         |  10 --
 src/ebs_backup.egg-info/top_level.txt        |   1 -
 src/main.py                                  |  40 ++++++
 8 files changed, 42 insertions(+), 213 deletions(-)
 delete mode 100644 src/ebs_backup.egg-info/PKG-INFO
 delete mode 100644 src/ebs_backup.egg-info/SOURCES.txt
 delete mode 100644 src/ebs_backup.egg-info/dependency_links.txt
 delete mode 100644 src/ebs_backup.egg-info/entry_points.txt
 delete mode 100644 src/ebs_backup.egg-info/requires.txt
 delete mode 100644 src/ebs_backup.egg-info/top_level.txt


hooks/post-receive
-- 
CS615 EBS-BACKUP; backup a directory into Elastic Block Storage (EBS)


More information about the cs615asa mailing list