Get online data using HTTP python interface in Compose

作者: ganhai 分类: 数学 发布时间: 2017-08-03 20:11 ė 184 6 没有评论

Get online data using HTTP python interface in Compose

Data aquisition online

In this part we’ll grab some data from Chicago public data portal, it serves large amount of data soruces in a variaty of domains, including administration, buildings, education, environment, etc. We’ll do some statistics about the schools in Chicago city, find the best 5 schools in math grading, parent engagement score.
Since the data source is provided on web for public download, we will leverage the Requests python package to download it. It is a popular HTTP library for Python to send web requests and handle responses.For detailed documentation of usage, please visit Requests docs.

First, in solidThinking Compose, create a new Python file, and enter following codes.

[/crayon]

Next, you’ll find the CSV data table is created at prescribed location. Open this file with Microsoft Excel, it looks like
Get online data using HTTP python interface in Compose

Note: if there are some errors from requests.get call in python command window, it could be network issue, just try a few more times.

Processing data with Compose

With the CSV data file, the next is using Compose to read the data and find out the information to our interests.
Run this command to read the csv file in Compose OML command window.

[/crayon]

We’ll look for schools with good math in grade 3-5, from excel we found the header lable Gr3-5 Grade Level Math % at column AM, in Compose we need to get the actual column index by running these commands.

[/crayon]

It displays the column index 39, so we wi’ll display all shcools with grade level over 80%.

[/crayon]

The results are displayed

[/crayon]

If we want to display the best 5 schools in math grades, following scripting will give the desired results.

[/crayon]

[/crayon]

Similarly, running following script gives the high scores of parent engagement among all shcools.

[/crayon]

[/crayon]

Processing data with Python pandas

Installation

pandas is one of the most popular package for data analysis in python community. It is a Python package providing fast, flexible, and expressive data structures designed to make working with structured (tabular, multidimensional, potentially heterogeneous) and time series data both easy and intuitive. By default, solidThinking Compose doesn’t come with a preinstalled pandas package. So next you’ll be shown how to install this package from scratch.
Assume you have VS 2013 Express installed. From Windows Start menu, click the shortcut VS2013 x64 Cross Tools Command Prompt to open the x64 target development command prompt. In the dos prompt, you have the windows SDK environment set up properly so the pandas source codes can be downloaded and compiled correctly on your machine.
In the dos prompty, run following command to overwrite the default Visual Studio version 2008 to 2013, then use pip command to fetch the pandas packages from Pypi official repository.

[/crayon]

After a few minutes when you see these message appears in the dos prompt, that means you have pandas installed successfully.

Successfully installed pandas-0.20.3

Processing data

To read the CSV file into Python with pandas, run the commands in python console

[/crayon]

It displays the results as an object of pandas data types.

[/crayon]

Now displays the first 5 schools of highest scores in math.

[/crayon]

[/crayon]

Now the data can be converted to list and prepared for further use in Compose

[/crayon]

Run following commands in Compose OML window, it will fetch the data from Python workspace to Compose OML workspace.

[/crayon]

Now the data we get in Compose from Python is ready for you for doing further analysis and post-processing.

本文出自扉启博客,转载时请注明出处及相应链接。

本文永久链接: https://www.feiqy.com/get-online-data-using-http-python-interface-in-compose/

发表评论

电子邮件地址不会被公开。 必填项已用*标注

Ɣ回顶部