1. 程式人生 > >Changing Python work to the cloud

Changing Python work to the cloud

Changing Python work to the cloud

My objectif is running my work Jupyter Notebook on Google Cloud Platform. After some tests using my machine to process deep learning using python, and medium data to process, its obvious that i need or to buy another computer ou migrate my work to the cloud. The cloud its my first option, rather than spending my money on a new GPU based laptop, and could be free on

Google Cloud Colaboratory.

Google has a platform very flexible to develop and run Python notebooks (Colaboratory) . The platform can connect to read or write data from your computer, or connect to your Drive (Google Drive), that its very good because you can construct all your development in the Google Cloud. The platform has to a connection with GitHub that provide you run your Python notebooks saved in Github or at the end save the copies of your work in your GitHub.

Colaboratory is a free Jupyter notebook environment that requires no setup and runs entirely in the cloud. Functionalities:

>> Colaboratory allows you to execute TensorFlow code in your browser;

>> Colaboratory notebook provides recipes for loading and saving data from external sources.

>> Offer a

Seedbank; a place to discover interactive machine learning examples;

>> Connection with GitHub. To load a specific notebook from github, append the github path.

>> Can importing a library that is not in Colaboratory. To import a library that’s not in Colaboratory by default, you can use !pip install or !apt-get install.

>> Charting in Colaboratory. A common use for notebooks is data visualization using charts. Colaboratory makes this easy with several charting tools available as Python imports.

To test and research the functionalities that platform offer, i made some basic operations reading Excel files from my Google Drive, and plot informations. I used a table sheet with informations about territories (or countries) that have population, PIB (intern product),and area of the territory. The file could be download here.

Lets start some examples.

Enter Bellow.

Can create new python notebook

Or can read your Notebook Python from Github:

I had create a new Python notebook, that you can consult here, on GitHub. I save them in my Google Drive.

The system created automaticaly a new folder in my Drive, named Colab Notebooks.

You can connect to the host, or in a local machine.

First i made a connection with my Drive, to read and write files.

from google.colab import drive
# Obtain autorization to acess you google drive
drive.mount(‘/content/gdrive’)

We need obtain authorizations from google drive, reading the key access in the that link . You copy the key (CTRL C / CTRL V) to the input Authorization code, like image below.

Reading a file from Drive.

Reading files using Pandas

As we see it`s the same form to read here, like we do in Python. The difference is the folder name.

I created a path variable to use in all my files reads.

An interesting functionality it’s you can make comments in your code. The same functionality you have in Google Sheets, its here.

New packages could be installed using !pip command.

And now lets go made some graphs. First import necessary packages.

Read files. I putted my excel files in my Google drive. The file it`s that.

Cread a Def to made a bar Graph.

Let’s see the most populous territory in the world.

As we see all the commands are normal Python, like we do in Python command line ou in Jupyter Notebook. That it`is good because we don’t need adapt code to run in the platform.

Let’s see the Less populous territory in the world

Plotting Area (Km2) by Territory. Biggest 30 bigs territories.

Analysing PIB by Territory

Analysing Population by Territory

Plotting all the statistics

Conclusion: The platform is an excellent form to develop your work using Python, very consistent, fast and integrated with the common develope tools that we have today. It’s simple and free!

Some references:

How use colabratec ( GPU free machine from Google):

How to read tables from Google drive:

相關推薦

Changing Python work to the cloud

Changing Python work to the cloudMy objectif is running my work Jupyter Notebook on Google Cloud Platform. After some tests using my machine to process dee

From sensor to the Cloud in just 15 lines of Python

The world of IoT is all about moving data to the cloud and then back to the devices. That process may sound simple, but once you start doing it, yo

AWS Storage Gateway: Virtual Tape Backup to the Cloud

The AWS Storage Gateway service can be configured to act as a Virtual Tape Library (VTL) that spans from your on-premises environment, where your

How to migrate petabyte-scale data to the cloud

When planning on getting your data into the cloud as a part of an application migration or a data center shutdown, you will run into the

Massive Data Transfer to the Cloud

Introducing the Snow family The Snow family of services offers a number of physical devices and capacity points, including some with built

svn: E155021: This client is too old to work with the working copy at..... No changes detected

伺服器svn版本1.6+ ,IDEA版本2017.1.5 java JDK 1.8.0_91 網上搜到大部分是讓upgrade,但是眾所周知Idea是死活找不到upgrade鍵的,思索還是版本不對應的原因,最終解決辦法是: 頻繁的試,從svn版本1.9.+一直往下試,試到1.6.+,最

關於yum安裝出現You could try using --skip-broken to work around the problem You could try running: rpm -Va --nofiles --nodigest解決方法

[[email protected] ~]# yum -y localinstall zabbix-proxy-mysql-4.0.0-2.el7.x86_64.rpmLoaded plugins: fastestmirror, securitySetting up Local Package Pr

[Python] send text to the Mac clipboard

pip install clipboard A cross platform clipboard operation library of Python. Works for Windows, Mac and Linux. Well, as I was trying to implement this,

Utilising machine learning, AI and the cloud to deliver a more personalised customer experience

Paul Armstrong, Principal Solutions Architect, Amazon Web Services: "To deliver a truly personalised travel experience, the first thing airports and airlin

Australia's Crime Stoppers to digitise crime reporting through the cloud

Australia's criminal information reporting service Crime Stoppers has announced it will be implementing a new online reporting tool, extending to members o

A Virtual Jewish Nation May Teach the World How to Live on the Cloud

A Virtual Jewish Nation May Teach the World How to Live on the CloudWe’re all familiar with “the cloud.” But have heard about “cloud nations”?If you ask fu

An introduction to IBM Cloud Log Analysis with the IBM Cloud Kubernetes Service

IBM Cloud Log Analysis is a service which plugs directly into the IBM Cloud Kubernetes Service, allowing for cluster-level log aggreg

Companies Leveraging The Cloud’s Best Parts are 23x More Likely to be High Performers, says Google

Companies Leveraging The Cloud’s Best Parts are 23x More Likely to be High Performers, says GoogleRestonLogic is Uniquely Positioned to Unlock the Competit

Carbon Farmers Work to Clean Up the World’s Mess

This story is for Medium members.Continue with FacebookContinue with GoogleMedium curates expert stories from leading publishers exclusively for members (w

Pipenv: A Guide to the New Python Packaging Tool

Pipenv is a packaging tool for Python that solves some common problems associated with the typical workflow using pip, virtualenv, and the good old requ

Contributing to the AWS Cloud Development Kit

This blog was authored by Mike Cowgill, Principal Engineer at Intuit (maker of TurboTax and QuickBooks) and active collaborator in the AWS Cloud D

Move data into the cloud up to 300% faster

Amazon Web Services is Hiring. Amazon Web Services (AWS) is a dynamic, growing business unit within Amazon.com. We are currently hiring So

How to provision desktops in the cloud

In this project, you’ll learn how to deploy cloud desktops using Amazon WorkSpaces and the AWS Directory Service. Amazon WorkSpaces is a

Connecting FFmpeg Using RTP to AWS Media Services in the Cloud

In this third installment of a five-part blog series, you are learning how to configure and connect various encoders to AWS Media Services. AWS Me

Announcing Amazon FreeRTOS – Enabling Billions of Devices to Securely Benefit from the Cloud

I was recently reading an article on ReadWrite.com titled “IoT devices go forth and multiply, to increase 200% by 2021“, and while the article not