[cs615asa] [git commit] CS615 EBS-BACKUP; backup a directory into Elastic Block Storage (EBS) branch main updated. 8921b4e46c664e4dcf60035980e4af77dada5e61

Git Owner jschauma at stevens.edu
Sun May 9 21:41:59 EDT 2021


This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "CS615 EBS-BACKUP; backup a directory into Elastic Block Storage (EBS)".

The branch, main has been updated
       via  8921b4e46c664e4dcf60035980e4af77dada5e61 (commit)
       via  73b37ab14f53ce8a96bfb95110c438d735057e99 (commit)
       via  817b28a95c284499b0a80563985b61a55712c015 (commit)
       via  8dcf91bf17d0c656580285f5b83f8cff4834de24 (commit)
      from  a189c8ecdbb0c6e3e19931449af49fa82f467bd2 (commit)

Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.

- Log -----------------------------------------------------------------
commit 8921b4e46c664e4dcf60035980e4af77dada5e61
Merge: a189c8e 73b37ab
Author: Charles Magyar IV <gartril854 at gmail.com>
Date:   Sun May 9 21:41:15 2021 -0400

    I (Charles) am merging Dave's code.  The man page is stylized and can be installed via pip3 install using sudo.
    
    Merge commit '73b37ab14f53ce8a96bfb95110c438d735057e99' of cs615asa:~ddelaus/ebs-backup into main


commit 73b37ab14f53ce8a96bfb95110c438d735057e99
Merge: 817b28a 8dcf91b
Author: Dave DeLaus <ddelaus at stevens.edu>
Date:   Sun May 9 20:42:41 2021 -0400

    Merge branch 'main' of cs615asa:ebs-backup into main


commit 817b28a95c284499b0a80563985b61a55712c015
Author: Dave DeLaus <ddelaus at stevens.edu>
Date:   Sun May 9 20:36:38 2021 -0400

    manpage support added to the package, readme also updated

diff --git a/ebs-backup.1.gz b/ebs-backup.1.gz
new file mode 100644
index 0000000..aad9ca0
Binary files /dev/null and b/ebs-backup.1.gz differ
diff --git a/setup.py b/setup.py
index 02ef492..1a722ac 100644
--- a/setup.py
+++ b/setup.py
@@ -1,6 +1,7 @@
 #!/usr/bin/env python
 
 from setuptools import setup, find_namespace_packages
+import shutil
 
 with open("ebs-backup.txt", "r", encoding="utf-8") as fh:
     long_description = fh.read()
@@ -8,7 +9,14 @@ with open("ebs-backup.txt", "r", encoding="utf-8") as fh:
 with open("src/requirements.txt", "r", encoding="utf-8") as fh:
     requirements = fh.read()
     all_requirements = [req.split("==")[0] for req in requirements.split("\n")]
-
+try:
+	shutil.copyfile('ebs-backup.1.gz', '/usr/share/man/man1/ebs-backup.1.gz')
+except:
+	print("run with sudo to install manpage")
+try:
+	shutil.copyfile('ebs-backup.1.gz', '/usr/local/share/man/man1/ebs-backup.1.gz')
+except:
+	print("run with sudo to install manpage")	
 setup (
     name = 'ebs-backup',
     description = 'Tool to backup ebs volumes',
@@ -31,4 +39,4 @@ setup (
         "License :: OSI Approved :: MIT License",
         "Programming Language :: Python :: 3.7",
     ]
-)
\ No newline at end of file
+)

commit 8dcf91bf17d0c656580285f5b83f8cff4834de24
Author: David DeLaus <ddelaus at stevens.edu>
Date:   Sun May 9 15:09:05 2021 -0400

    manpage testing

diff --git a/ebs-backup.1 b/ebs-backup.1
new file mode 100644
index 0000000..9e5c717
--- /dev/null
+++ b/ebs-backup.1
@@ -0,0 +1,172 @@
+.TH EBS-BACKUP 1
+
+.SH NAME
+     ebs-backup -- backup a directory into Elastic Block Storage (EBS)
+
+.SH SYNOPSIS
+     ebs-backup [-h] [-l filter] [-r filter] [-v volume-id] dir
+
+.SH DESCRIPTION
+     The ebs-backup tool performs a backup of the given directory into Amazon
+     Elastic Block Storage (EBS).  This is achieved by creating a volume of
+     the appropriate size, attaching it to an EC2 instance and finally copying
+     the files from the given directory onto this volume.
+
+.SH OPTIONS
+     ebs-backup accepts the following command-line flags:
+
+     -h		   Print a usage statement and exit.
+
+     -l filter	   Pass data through the given filter command on the local
+		   host before copying the data to the remote system.
+
+     -r filter	   Pass data through the given filter command on the remote
+		   host before writing the data to the volume.
+
+     -v volume-id  Use the given volume instead of creating a new one.
+
+.SH DETAILS
+     ebs-backup will perform a backup of the given directory to an ESB volume.
+     The backup is done with the help of the tar(1) command on the local host,
+     writing the resulting archive directly to the block device.  That is,
+     ebs-backup does not create or use a filesystem on the volume.  Instead,
+     ebs-backup utilizes the dd(1) command to write out the data to the raw
+     volume.  In essence, ebs-backup wraps the following pipeline:
+
+	   tar cf - <dir> [| local-filter] | ssh ec2-instance "[remote-filter |] dd of=/dev/xbd2d"
+
+     Here, "/dev/xbd2d" stands for the suitable raw disk device, which may
+     differ depending on the instance type.
+
+     ebs-backup does not use any temporary files, nor creates a local copy of
+     the archive it writes to the volume.
+
+     ebs-backup can pass the archive it creates through a filter command on
+     either the local or the remote host.  This allows the user to e.g. per-
+     form encryption of the backup.
+
+     Unless the -v flag is specified, ebs-backup will create a new volume, the
+     size of which will be at least two times the size of the directory to be
+     backed up.
+
+     ebs-backup will create an instance suitable to perform the backup, attach
+     the volume in question and then back up the data from the given direc-
+     tory.  Afterwards, ebs-backup will terminate the instance it created.
+
+     ebs-backup will not create or modify any other AWS resources.
+
+.SH VERIFICATION
+     By default, ebs-backup simply writes the data to the volume.  To verify
+     that this was successful, the user may manually perform the following
+     tasks:
+
+	   aws ec2 run-instances
+	   aws ec2 attach-volume
+	   ssh instance "dd if=/dev/xbd2d" | tar tvf -
+
+.SH OUTPUT
+     If successful, ebs-backup will print the volume-id of the volume to which
+     it backed up the data as the only output.
+
+     Unless the EBS_BACKUP_VERBOSE environment variable is set, ebs-backup
+     will not generate any other output unless any errors are encountered.  If
+     that variable is set, it may print out some useful information about what
+     steps it is currently performing.
+
+     Any errors encountered cause a meaningful error message to be printed to
+     STDERR.
+
+.SH ENVIRONMENT
+     ebs-backup assumes that the user has set up their environment for general
+     use with the EC2 tools and ssh(1) without any special flags on the com-
+     mand-line.	 That is, the user has a suitable section in their ~/.ssh/con-
+     fig file to ensure that running 'ssh ec2-instance.amazonaws.com' suc-
+     ceeds.
+
+     To accomplish this, the user has created an SSH key pair named 'ebs-
+     backup' and configured their SSH setup to use that key to connect to EC2
+     instances.
+
+     Therefor, ebs-backup will not set nor modify the variables AWS_CON-
+     FIG_FILE, EC2_CERT, EC2_HOME or EC2_PRIVATE_KEY.
+
+     ebs-backup allows the user to add custom flags to the commands related to
+     starting a new EC2 instance via the EBS_BACKUP_FLAGS_AWS environment
+     variable.
+
+     ebs-backup also assumes that the user has set up their ~/.ssh/config file
+     to access instances in EC2 via ssh(1) without any additional settings.
+     It does allow the user to add custom flags to the ssh(1) commands it
+     invokes via the EBS_BACKUP_FLAGS_SSH environment variable.
+
+     As noted above, the EBS_BACKUP_VERBOSE variable may cause ebs-backup to
+     generate informational output as it runs.
+
+.SH EXIT STATUS
+     The ebs-backup will exit with a return status of 0 under normal circum-
+     stances.  If an error occurred, ebs-backup will exit with a value >0.
+
+.SH EXAMPLES
+     The following examples illustrate common usage of this tool.
+
+     To back up the entire filesystem:
+
+	   $ ebs-backup /
+	   vol-1a2b3c4d
+
+     To create a complete backup of the current working directory using
+     defaults to the volume with the ID vol-1a2b3c4d, possibly overwriting any
+     data previously stored there:
+
+	   $ ebs-backup -v vol-1a2b3c4d .
+	   vol-1a2b3c4d
+
+     To do the same thing again, but having the program tell us what it's
+     doing, the user can set the EC2_BACKUP_VERBOSE environment variable.
+     Possible diagnostic messages generated when that variable is set are
+     shown below:
+
+	   $ EBS_BACKUP_VERBOSE=1 ebs-backup -v vol-1a2b3c4d .
+	   Verifying volume...
+	   Volume vol-1a2b3c4d is in availability zone 'us-east-1a'.
+	   Creating a suitable instance in 'us-east-1a'...
+	   Attaching volume vol-1a2b3c4d to instance i-123abcd456...
+	   Performing backup...
+	   Terminating instance i-123abcd456...
+	   Backup complete, 1.7 GB of data written to:
+	   vol-1a2b3c4d
+
+     Suppose a user has their ~/.ssh/config set up to use the private key
+     ~/.ec2/stevens but wishes to use the key ~/.ssh/ec2-key instead:
+
+	   $ export EBS_BACKUP_FLAGS_SSH="-i ~/.ssh/ec2-key"
+	   $ ebs-backup .
+	   vol-1a2b3c4d
+
+     To force creation of an instance type of t1.micro instead of whatever
+     defaults might apply
+
+	   $ export EBS_BACKUP_FLAGS_AWS="--instance-type t1.micro"
+	   $ ebs-backup .
+	   vol-1a2b3c4d
+
+     To locally encrypt the backup of the '/var/secrets' directory:
+
+	   $ ebs-backup -l 'gpg -e -r 9BED3DD7' /var/secrets
+	   vol-1a2b3c4d
+
+     The same as above, but perform encryption on the remote system:
+
+	   $ ebs-backup -r 'gpg -e -r 9BED3DD7' /var/secrets
+	   vol-1a2b3c4d
+
+.SH SEE ALSO
+     aws help, cat(1), dd(1), ssh(1), tar(1)
+
+.SH HISTORY
+     ebs-backup was originally assigned by Jan Schaumann
+     <jschauma at cs.stevens.edu> as a homework assignment for the class "Aspects
+     of System Administration" at Stevens Institute of Technology in the
+     Spring of 2011.
+
+
diff --git a/setup.py b/setup.py
index 02ef492..0474bfb 100644
--- a/setup.py
+++ b/setup.py
@@ -27,8 +27,12 @@ setup (
     },
     license="MIT",
     author_email="hdaly1 at stevens.edu",
+    data_files = [('man/man1',['ebs-backup.1'])],
+    package_data = {'': ['*.txt',
+                         'examples/*.py',
+                         'examples/*/*.py']},
     classifiers=[
         "License :: OSI Approved :: MIT License",
         "Programming Language :: Python :: 3.7",
     ]
-)
\ No newline at end of file
+)
diff --git a/src/ebs_backup.egg-info/PKG-INFO b/src/ebs_backup.egg-info/PKG-INFO
new file mode 100644
index 0000000..456b8c7
--- /dev/null
+++ b/src/ebs_backup.egg-info/PKG-INFO
@@ -0,0 +1,184 @@
+Metadata-Version: 1.2
+Name: ebs-backup
+Version: 0.0.1
+Summary: Tool to backup ebs volumes
+Home-page: UNKNOWN
+Author: 2021S CS615
+Author-email: hdaly1 at stevens.edu
+License: MIT
+Description: EBS-BACKUP(1)		NetBSD General Commands Manual		 EBS-BACKUP(1)
+        
+        NAME
+             ebs-backup -- backup a directory into Elastic Block Storage (EBS)
+        
+        SYNOPSIS
+             ebs-backup [-h] [-l filter] [-r filter] [-v volume-id] dir
+        
+        DESCRIPTION
+             The ebs-backup tool performs a backup of the given directory into Amazon
+             Elastic Block Storage (EBS).  This is achieved by creating a volume of
+             the appropriate size, attaching it to an EC2 instance and finally copying
+             the files from the given directory onto this volume.
+        
+        OPTIONS
+             ebs-backup accepts the following command-line flags:
+        
+             -h		   Print a usage statement and exit.
+        
+             -l filter	   Pass data through the given filter command on the local
+        		   host before copying the data to the remote system.
+        
+             -r filter	   Pass data through the given filter command on the remote
+        		   host before writing the data to the volume.
+        
+             -v volume-id  Use the given volume instead of creating a new one.
+        
+        DETAILS
+             ebs-backup will perform a backup of the given directory to an ESB volume.
+             The backup is done with the help of the tar(1) command on the local host,
+             writing the resulting archive directly to the block device.  That is,
+             ebs-backup does not create or use a filesystem on the volume.  Instead,
+             ebs-backup utilizes the dd(1) command to write out the data to the raw
+             volume.  In essence, ebs-backup wraps the following pipeline:
+        
+        	   tar cf - <dir> [| local-filter] | ssh ec2-instance "[remote-filter |] dd of=/dev/xbd2d"
+        
+             Here, "/dev/xbd2d" stands for the suitable raw disk device, which may
+             differ depending on the instance type.
+        
+             ebs-backup does not use any temporary files, nor creates a local copy of
+             the archive it writes to the volume.
+        
+             ebs-backup can pass the archive it creates through a filter command on
+             either the local or the remote host.  This allows the user to e.g. per-
+             form encryption of the backup.
+        
+             Unless the -v flag is specified, ebs-backup will create a new volume, the
+             size of which will be at least two times the size of the directory to be
+             backed up.
+        
+             ebs-backup will create an instance suitable to perform the backup, attach
+             the volume in question and then back up the data from the given direc-
+             tory.  Afterwards, ebs-backup will terminate the instance it created.
+        
+             ebs-backup will not create or modify any other AWS resources.
+        
+        VERIFICATION
+             By default, ebs-backup simply writes the data to the volume.  To verify
+             that this was successful, the user may manually perform the following
+             tasks:
+        
+        	   aws ec2 run-instances
+        	   aws ec2 attach-volume
+        	   ssh instance "dd if=/dev/xbd2d" | tar tvf -
+        
+        OUTPUT
+             If successful, ebs-backup will print the volume-id of the volume to which
+             it backed up the data as the only output.
+        
+             Unless the EBS_BACKUP_VERBOSE environment variable is set, ebs-backup
+             will not generate any other output unless any errors are encountered.  If
+             that variable is set, it may print out some useful information about what
+             steps it is currently performing.
+        
+             Any errors encountered cause a meaningful error message to be printed to
+             STDERR.
+        
+        ENVIRONMENT
+             ebs-backup assumes that the user has set up their environment for general
+             use with the EC2 tools and ssh(1) without any special flags on the com-
+             mand-line.	 That is, the user has a suitable section in their ~/.ssh/con-
+             fig file to ensure that running 'ssh ec2-instance.amazonaws.com' suc-
+             ceeds.
+        
+             To accomplish this, the user has created an SSH key pair named 'ebs-
+             backup' and configured their SSH setup to use that key to connect to EC2
+             instances.
+        
+             Therefor, ebs-backup will not set nor modify the variables AWS_CON-
+             FIG_FILE, EC2_CERT, EC2_HOME or EC2_PRIVATE_KEY.
+        
+             ebs-backup allows the user to add custom flags to the commands related to
+             starting a new EC2 instance via the EBS_BACKUP_FLAGS_AWS environment
+             variable.
+        
+             ebs-backup also assumes that the user has set up their ~/.ssh/config file
+             to access instances in EC2 via ssh(1) without any additional settings.
+             It does allow the user to add custom flags to the ssh(1) commands it
+             invokes via the EBS_BACKUP_FLAGS_SSH environment variable.
+        
+             As noted above, the EBS_BACKUP_VERBOSE variable may cause ebs-backup to
+             generate informational output as it runs.
+        
+        EXIT STATUS
+             The ebs-backup will exit with a return status of 0 under normal circum-
+             stances.  If an error occurred, ebs-backup will exit with a value >0.
+        
+        EXAMPLES
+             The following examples illustrate common usage of this tool.
+        
+             To back up the entire filesystem:
+        
+        	   $ ebs-backup /
+        	   vol-1a2b3c4d
+        
+             To create a complete backup of the current working directory using
+             defaults to the volume with the ID vol-1a2b3c4d, possibly overwriting any
+             data previously stored there:
+        
+        	   $ ebs-backup -v vol-1a2b3c4d .
+        	   vol-1a2b3c4d
+        
+             To do the same thing again, but having the program tell us what it's
+             doing, the user can set the EC2_BACKUP_VERBOSE environment variable.
+             Possible diagnostic messages generated when that variable is set are
+             shown below:
+        
+        	   $ EBS_BACKUP_VERBOSE=1 ebs-backup -v vol-1a2b3c4d .
+        	   Verifying volume...
+        	   Volume vol-1a2b3c4d is in availability zone 'us-east-1a'.
+        	   Creating a suitable instance in 'us-east-1a'...
+        	   Attaching volume vol-1a2b3c4d to instance i-123abcd456...
+        	   Performing backup...
+        	   Terminating instance i-123abcd456...
+        	   Backup complete, 1.7 GB of data written to:
+        	   vol-1a2b3c4dgg
+        
+             Suppose a user has their ~/.ssh/config set up to use the private key
+             ~/.ec2/stevens but wishes to use the key ~/.ssh/ec2-key instead:
+        
+        	   $ export EBS_BACKUP_FLAGS_SSH="-i ~/.ssh/ec2-key"
+        	   $ ebs-backup .
+        	   vol-1a2b3c4d
+        
+             To force creation of an instance type of t1.micro instead of whatever
+             defaults might apply
+        
+        	   $ export EBS_BACKUP_FLAGS_AWS="--instance-type t1.micro"
+        	   $ ebs-backup .
+        	   vol-1a2b3c4d
+        
+             To locally encrypt the backup of the '/var/secrets' directory:
+        
+        	   $ ebs-backup -l 'gpg -e -r 9BED3DD7' /var/secrets
+        	   vol-1a2b3c4d
+        
+             The same as above, but perform encryption on the remote system:
+        
+        	   $ ebs-backup -r 'gpg -e -r 9BED3DD7' /var/secrets
+        	   vol-1a2b3c4d
+        
+        SEE ALSO
+             aws help, cat(1), dd(1), ssh(1), tar(1)
+        
+        HISTORY
+             ebs-backup was originally assigned by Jan Schaumann
+             <jschauma at cs.stevens.edu> as a homework assignment for the class "Aspects
+             of System Administration" at Stevens Institute of Technology in the
+             Spring of 2011.
+        
+        NetBSD 8.0		       February 7, 2021			    NetBSD 8.0
+Platform: UNKNOWN
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3.7
+Requires-Python: >=3.7
diff --git a/src/ebs_backup.egg-info/SOURCES.txt b/src/ebs_backup.egg-info/SOURCES.txt
new file mode 100644
index 0000000..715c026
--- /dev/null
+++ b/src/ebs_backup.egg-info/SOURCES.txt
@@ -0,0 +1,12 @@
+README
+ebs-backup.1
+ebs-backup.1.gz
+setup.py
+src/ebs_backup.egg-info/PKG-INFO
+src/ebs_backup.egg-info/SOURCES.txt
+src/ebs_backup.egg-info/dependency_links.txt
+src/ebs_backup.egg-info/entry_points.txt
+src/ebs_backup.egg-info/requires.txt
+src/ebs_backup.egg-info/top_level.txt
+test/test_arg_parsing.py
+test/test_env_parsing.py
\ No newline at end of file
diff --git a/src/ebs_backup.egg-info/dependency_links.txt b/src/ebs_backup.egg-info/dependency_links.txt
new file mode 100644
index 0000000..8b13789
--- /dev/null
+++ b/src/ebs_backup.egg-info/dependency_links.txt
@@ -0,0 +1 @@
+
diff --git a/src/ebs_backup.egg-info/entry_points.txt b/src/ebs_backup.egg-info/entry_points.txt
new file mode 100644
index 0000000..c46fe94
--- /dev/null
+++ b/src/ebs_backup.egg-info/entry_points.txt
@@ -0,0 +1,3 @@
+[console_scripts]
+ebs-backup = main:main
+
diff --git a/src/ebs_backup.egg-info/requires.txt b/src/ebs_backup.egg-info/requires.txt
new file mode 100644
index 0000000..f0dbda1
--- /dev/null
+++ b/src/ebs_backup.egg-info/requires.txt
@@ -0,0 +1,10 @@
+boto3
+botocore
+environs
+jmespath
+marshmallow
+python-dateutil
+python-dotenv
+s3transfer
+six
+urllib3
diff --git a/src/ebs_backup.egg-info/top_level.txt b/src/ebs_backup.egg-info/top_level.txt
new file mode 100644
index 0000000..a9a5aec
--- /dev/null
+++ b/src/ebs_backup.egg-info/top_level.txt
@@ -0,0 +1 @@
+tmp

-----------------------------------------------------------------------

Summary of changes:
 ebs-backup.txt => ebs-backup.1               | 344 +++++++++++++--------------
 ebs-backup.1.gz                              | Bin 0 -> 2557 bytes
 setup.py                                     |  16 +-
 src/ebs_backup.egg-info/PKG-INFO             | 184 ++++++++++++++
 src/ebs_backup.egg-info/SOURCES.txt          |  12 +
 src/ebs_backup.egg-info/dependency_links.txt |   1 +
 src/ebs_backup.egg-info/entry_points.txt     |   3 +
 src/ebs_backup.egg-info/requires.txt         |  10 +
 src/ebs_backup.egg-info/top_level.txt        |   1 +
 9 files changed, 397 insertions(+), 174 deletions(-)
 copy ebs-backup.txt => ebs-backup.1 (93%)
 create mode 100644 ebs-backup.1.gz
 create mode 100644 src/ebs_backup.egg-info/PKG-INFO
 create mode 100644 src/ebs_backup.egg-info/SOURCES.txt
 create mode 100644 src/ebs_backup.egg-info/dependency_links.txt
 create mode 100644 src/ebs_backup.egg-info/entry_points.txt
 create mode 100644 src/ebs_backup.egg-info/requires.txt
 create mode 100644 src/ebs_backup.egg-info/top_level.txt


hooks/post-receive
-- 
CS615 EBS-BACKUP; backup a directory into Elastic Block Storage (EBS)


More information about the cs615asa mailing list