Setting up your MAC OSX 11 to use the AWS CLI and other python tools for accessing Amazon Web Services (AWS)
Every time that I change work laptops or upgrade my laptop to a new version of MAC OSX, I need to remove and reconfigure parts of MAC Ports and get things going. This is a short guide to getting things working for new OSX users working with the Amazon Command Line Interface (AWS CLI).
First step is to make sure that you have working credentials for a user account. For this we will need to create some IAM objects in your AWS Account. If you have root access or its equivalent, please login with those credentials. We will create the following objects:
Group named “ReadOnlyAdmin” <- Assigned RO Admin level credentials.
User named “roadmin” <- Added to Group
Keys for the User “roadmin”
Choose Create New Group
Enter Security Audit In the Policy and select it.
Create the Group
Now we create the User and add the user to the group.
Very important. Download the keys now. They will not be available later.
Very important. Download the keys now. They will not be available later.
At this point you should have the following handy:
- Username
- Access Key
- Secret Key
- Default Region (us-east-2)
(Where are you working, us-east-1, us-east-2, us-west-1 and so on). I am working in us-east-2 (so that will be my default.
Installation of MAC Ports for Python, Jupyter, and AWS CLI
- Download the MAC Ports installer from: https://github.com/macports/macports-base/releases/
Setup the environment variables
Mac PORTS binaries and man pages are located under the /opt/local directory. There are two ways to accomplish setting the variables automatically. You can add them to the users profile or the global profiles. I prefer to add things globally so that other users can also use them. We need to set the following variables:
- PATH
- MANPATH
Run this command in the terminal window to create the appropriate path. You will need to close your terminal window and reopen to get the new changes after.
Adding the binaries after system binaries use this method:
sudo sh -c “echo “/opt/local/bin” > /etc/paths.d/mac-ports”
You can test if it worked by running this command
env | grep PATH
Results:
PATH=/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/local/Library/Frameworks/Python.framework/Versions/3.8/bin:/opt/local/bin
Adding the binaries before system binaries use this method.
- Note: After updates or upgrades you may need to run this
sudo sed -i ‘’ -e $’1i\\\n/opt/local/bin’ /etc/paths
sudo sed -i ‘’ -e $’2i\\\n/opt/local/sbin’ /etc/paths
Add the MANPATH Variable to the system
sudo bash -c "echo 'MANPATH=/opt/local/share/man:\$MANPATH' > /etc/manpaths.d/mac-ports"
You are all set to have easy access to MacPorts now.
Next we will add all of the tools we will need for developing AWS Python applications or access it with Jupyter Notebook.
- Open the terminal window
- run:
sudo port install py38-awscli
— -> Computing dependencies for py38-awscli
The following dependencies will be installed:
awscli_select
bzip2
expat
gettext
libedit
libffi
libiconv
libyaml
ncurses
openssl
py38-asn1
py38-botocore
py38-certifi
py38-colorama
py38-dateutil
py38-docutils
py38-ipaddress
py38-jmespath
py38-roman
py38-rsa
py38-s3transfer
py38-setuptools
py38-six
py38-tz
py38-urllib3
py38-yaml
python38
python3_select
python_select
sqlite3
xz
zlib
Continue? [Y/n]: Y - Set the Python as primary if you so desire:
python38 has the following notes:
To make this the default Python or Python 3 (i.e., the version run by the ‘python’ or ‘python3’ commands), run one or both of: sudo port select --set python python38
sudo port select --set python3 python38- Install Jupyter Lab (aka Notebook)
sudo port install py38-jupyterlab
— -> Computing dependencies for py38-jupyterlab
The following dependencies will be installed:
ipython3_select
ipython_select
py38-appnope
py38-argon2-cffi
py38-async_generator
py38-attrs
py38-backcall
py38-bleach
py38-cffi
py38-chardet
py38-decorator
py38-defusedxml
py38-entrypoints
py38-idna
py38-ipykernel
py38-ipython
py38-ipython_genutils
py38-jedi
py38-jinja2
py38-json5
py38-jsonschema
py38-jupyter_client
py38-jupyter_core
py38-jupyterlab_pygments
py38-jupyterlab_server
py38-markupsafe
py38-mistune
py38-nbclient
py38-nbconvert
py38-nbformat
py38-nest_asyncio
py38-notebook
py38-packaging
py38-pandocfilters
py38-parsing
py38-parso
py38-pexpect
py38-pickleshare
py38-prometheus_client
py38-prompt_toolkit
py38-ptyprocess
py38-pycparser
py38-pygments
py38-pyrsistent
py38-requests
py38-send2trash
py38-terminado
py38-testpath
py38-tornado
py38-traitlets
py38-wcwidth
py38-webencodings
py38-zmq
pygments_select
zmq
— -> Some of the ports you installed have notes:
py38-ipython has the following notes:
To make this the default IPython or IPython3 (i.e., the version run by the ‘ipython’ or ‘ipython3’ commands), run one or both of:
sudo port select --set ipython py38-ipython
sudo port select --set ipython3 py38-ipython
py38-pygments has the following notes:
To make the Python 3.8 version of Pygments the one that is run when you execute the commands without a version suffix, e.g. ‘pygmentize’, run:
port select --set pygments py38-pygments
Configure AWS CLI for access to your AWS Account and test.
Configure:
From a terminal window run the following command. Remember to have your user keys handy.
$ aws configure --profile roadmins
AWS Access Key ID [None]: NZ7JN2HX3KC89PC79V09
AWS Secret Access Key [None]: !bAivY20SDQCFaB^A!G5VAy597va5ZufwVQGdab@
Default region name [None]: us-east-2
Default output format [None]: text
I am using a “profile” to avoid conflict with other user keys on my system. Each seperate user key pair can be a profile. If this is your first an only, you can omit the --profile part to configure the default profile.
Test:
Run this command to testaws --profile roadmins ec2 describe-vpcs
Results should be similar:
VPCS 172.31.0.0/16 dopt-06a3366d default True 971650852359 available vpc-5f318134
CIDRBLOCKASSOCIATIONSET vpc-cidr-assoc-7410a21f 172.31.0.0/16
CIDRBLOCKSTATE associated
VPCS 10.1.0.0/16 dopt-06a3366d default False 971650852359 available vpc-028ffd91330d05253
CIDRBLOCKASSOCIATIONSET vpc-cidr-assoc-0c4bf677e28d16205 10.1.0.0/16
CIDRBLOCKSTATE associated
TAGS Name atc-vpc01
** I am assuming this is a new account without EC2s and other objects. So we will get a list of available VPCs because that exists in every new account.
Additional Reading and recommendations
- Add your AWS ssh keys to .ssh/config and configure for easy access
- Using Jupyter Notebook for accessing the AWS Cli
- Additional tools that are useful in Jupyter notebook and MAC Ports: ImageMagick, Pillow, FFMEG, SQLite and any other linux tool you might want including tools like gnu versions of build-in tools (i.e. ggrep or gawk)