Category Archives: data in a file

STARTING WITH PYTHON3 – The very beginning – part 2

Journal: uffmm.org,
ISSN 2567-6458, July 9, 2019
Email: info@uffmm.org
Author: Gerd Doeben-Henisch
Email:
gerd@doeben-henisch.de

CONTEXT

This is the next step in the python3 programming project. The first step can be found here. The overall context is still the python Co-Learning project.

PROGRAMMING TOOLS

First SW-tools to use for programming

In this second session we extend the overview of the possible programming tools, how they are interrelated, and how they work.

In the figure above you can see the windows 10 operating system as the root system for everything else. The win10 system communicates with the PATH-variable and uses this information for many operations. How on can edit this variable has been shown in the last session.

One can activate directly from the win10 system the power-shell with a command-line interface. Entering the right code one can activate from the power-shell either directly a python-shell for python commands or one can activate other programs like the editor ‘notepad’ or ‘notepad++’. With such editors one can edit python scripts, store them, and then run these scripts from the power-shell by calling a python-shell with these scripts as arguments (as shown in the first session).

The python shell allows the direct entering of python commands and gives immediately feedback whether it works and how. Therefore one calls this an interactive shell which is very handy to check quickly some commands and their effects.

Another tool, which we will use in this session, is the integrated script environment (IDLE). This is like the python-shell but with some additional functionalities (see below). The main usage is for editing larger python scripts with a built-in editor and for running these scripts.

THE IDLE TOOL

To use this new tool you can press the windows button to see the list of all apps (programs) available on your computer. Under ‘P’ you will find python 3.7.3 and within python you will find an entry for IDLE. By selecting this item and clicking on the right mouse-button you can select the option to attach this icon to the task bar. If it is there you can use it.

If you start the IDLE tool by clicking on the icon from the task bar it opens as a new python interactive shell with some more options.

A first thing you can do is to ask for the actual path you are in. For this you have to import the python module ‘os’ (operating system) and use the command ‘getcwd()‘ from this module. Entering ‘os.getcwd()‘ in the python command line generates the actual path as output on the next line.

>>> import os
>>> os.getcwd()
‘C:\\Users\\gerd_2\\AppData\\Local\\Programs\\Python\\Python37-32’
>>>

This reveals that the actual path is pointing to the location of the python exe module (on my pc). This is not what I want because I have created in the first session a folder with name ‘code’ in my home directory ‘\Users\gerd_2’. From inside of the IDLE tool it is not possible to change the actual path.  But python as language provides lots of options to do this. One option is described below:

The module os offers several functions. Besides the function ‘os.getcwd()’ which we have used already there is another command ‘os.chdir(pathname)‘. But to directly change the actual path one has to be cautious because the path ‘C:\\Users\gerd_2\code‘ includes the ‘\’-sign, this cannot be read directly by the os.chdir() command. You can surround this problem by using the ‘\’-sign twice: first as an ‘escape sign’ and then as the ‘object sign’, resulting in the following command format: ‘C:\\Users\\gerd_2\\code‘. Entering this nothing is given as a result, and when you repeat the question ‘os.getcwd()’ you will receive as new answer the new path. Here the dialog with the python-shell:

Python 3.7.3 (v3.7.3:ef4ec6ed12, Mar 25 2019, 21:26:53) [MSC v.1916 32 bit (Intel)] on win32

Type “help”, “copyright”, “credits” or “license()” for more information.

>>> import os

>>> os.getcwd()

‘C:\\Users\\gerd_2\\AppData\\Local\\Programs\\Python\\Python37-32’

>>> os.chdir(‘C:\\Users\\gerd_2\\code’)

>>> os.getcwd()

‘C:\\Users\\gerd_2\\code’

>>>

You can see that the python command ‘os.getcwd() has been used twice. If you want to repeat some command you can call-back the command history of the python-shell with the keystrokes ‘ALT+P‘. This recalls the past (P) of the command history.

Comment: In the command

>> os.chdir(‘C:\\Users\\gerd_2\\code’)

I have used the back-slash sign ‘\’ twice to make the string fit as argument for the ‘os.chdir()’ command. As one can learn does python allow another solution, which looks like this:

>> os.chdir(r’C:\Users\gerd_2\code’)

The solution is to use an additional ‘r’ directly before the string ‘…’ telling the python interpreter that the following string has to be understood as a raw string. This works, try it out 🙂

IDLE AND EXECUTION OF A SCRIPT

Now if we are in the target folder for my scripts we can look to all files which are in this folder actually. For this we can use the python command ‘os.listdir()’:

>>> os.listdir()

[‘savesrc.txt’, ‘script1.py’, ‘script1.pyw’, ‘script1b.py’, ‘showargs.py’, ‘threenames.py’, ‘tst1.py’, ‘what.py’, ‘what2.py’, ‘__pycache__’]

>>>

You can detect in this list the python script ‘scrpt1.py’. Entering the name of this script either with .py extension or without will not enable an execution:

>>> script1.py

Traceback (most recent call last):

File “<pyshell#6>”, line 1, in <module>

script1.py

NameError: name ‘script1’ is not defined

From the first session we know that we can start the script within the power-shell directly. For this we have to activate the powershell, have to go into the desired folder ‘code’ …

PS C:\Users\gerd_2> cd code

PS C:\Users\gerd_2\code> dir

Verzeichnis: C:\Users\gerd_2\code

Mode LastWriteTime Length Name

—- ————- —— —-

d—– 04.07.2019 19:03 __pycache__

-a—- 01.07.2019 18:44 182 savesrc.txt

-a—- 01.07.2019 18:41 92 script1.py

-a—- 24.06.2019 23:23 126 script1.pyw

-a—- 24.06.2019 22:43 128 script1b.py

-a—- 04.07.2019 18:51 56 showargs.py

-a—- 28.06.2019 00:29 162 threenames.py

-a—- 24.06.2019 21:16 120 tst1.py

-a—- 24.06.2019 22:49 126 what.py

-a—- 24.06.2019 23:56 136 what2.py

… and then we can start the python-script ‘script1.py’:

PS C:\Users\gerd_2\code> python script1.py

win32

1267650600228229401496703205376

pythonpythonpythonpythonpythonpythonpythonpython

PS C:\Users\gerd_2\code>

But because we will here use the IDLE tool we proceed differently. We open the File-Menue to get the desired file script1.py:

Open file-directory for file search

Then we load  the python-script script1.py in the editor of the IDLE tool:

The text of the script

and then activate the RUN button for execution:

Activate the RUN button to execute the script

The script will then be executed and you will see the effect of the execution in the python shell. This looks the same as when you would have called the script within the power-shell calling  the python-shell.

There is still the other option to get the module running by the import command:

>>> import script1.py

win32

1267650600228229401496703205376

pythonpythonpythonpythonpythonpythonpythonpython

Traceback (most recent call last):

File “<pyshell#7>”, line 1, in <module>

import script1.py

ModuleNotFoundError: No module named ‘script1.py’; ‘script1’ is not a package

>>>

The import call works, but at the same time the python-shell states some error, that ‘script1.py’ is not recognized as a true module. This has to be clarified in the next session.

One possible continuation can be found HERE.

 

 

Example python3: pop0e – simple population program

eJournal: uffmm.org,
ISSN 2567-6458, 4-6.April 2019
Email: info@uffmm.org
Author: Gerd Doeben-Henisch
Email: gerd@doeben-henisch.de

CONTEXT

This is a possible 4th step in the overall topic ‘Co-Learning python3′. After downloading WinPython and activating the integrated editor ‘spyder’ (see here),  one can edit another simple program dealing with population dynamics in a most simple way (see the source code below under the title ‘EXAMPLE: pop0e.py’). This program is a continuation of the program pop0d.py, which has been described here.

COMMENTS

In this post I comment only on the changes between the actual program and the version before.

IMPORTS

In the new version one more liibrary is used for the handling of time stamps:

import time # Lib for time

STORING DATA IN A FILE

The only extension in the new version of the small program are some lines enabling the storage of the data from the simulation in a file.

data = np.column_stack((x,pop))

This line is formating the plot-values as x and x axes written as two columns bedides each other in a file.

What comes next is a construction of a file name which includes the actual time as well as the values of the br and the dr variable:

ts = time.gmtime()
t=time.strftime(“%c”, ts) # format time data into ISO format
t=t.replace(‘ ‘,’-‘)
t=t.replace(‘:’,’-‘)
header=’br=’+str(br)+’,’+’dr=’+str(dr)+’\n’+’X-Column, Y-Column’
fname=’hxyPTL’+t+’br=’+str(br)+’-‘+’dr=’+str(dr)

After this construction a file is generated with the plotting data as well as an expressive name.

np.savetxt(fname+’.txt’, data,header=header)

SOURCE CODE

#!/usr/bin/env python3
# -*- coding: utf-8 -*-
“””
Created on Thu 4-6 April Konga (Sweden) 2019

@author: gerd doeben-henisch
Email: gerd@doeben-henisch.de
“””

##################################
# pop0e()
###################################
#
# IDEA
#
# Simple program to compute the increase/ decrease of a population with
# the parameters population number (p), birth-rate (br) and death-rate (dr).
# In this version an extension with the following features:
# – a loop to repeat the computation for n-many cycles
# – a storage of the data in an array
# – an additional automatic storage of the plotting data in a file
# with the actual time in the file-name
# – a plot of the stored data for n-many cycles
# – the overall change of the population in %

#########################################################
# IMPORTS
# As part of the distribution of Winpython there are already many
# libraries pre-installed, which can be activated by the import command.
# Other libraries outside of the distribution have to be downloaded
# with the pip command
# Lib for plotting

import matplotlib.pyplot as plt # Lib for plotting
import numpy as np # Lib for math
import time # Lib for time

###########################################################
# DEFINITION

def pop0(p,br,dr):

p=p+(p*br)-(p*dr)

return p

###################################
# INPUT OF DATA

p = int(input(‘Population number ? ‘))

br = float(input(‘Birthrate in % ? ‘))
br = br/100

dr = float(input(‘Deathrate in % ? ‘))
dr = dr/100

n = int(input(‘How many cycles ? ‘))

baseYear = int(input(‘What is your Base Year ? ‘))

#############################################
# GLOBAL VARIABLES

pop = [] # storage for the pop-numbers for plotting
pop.append(p)

#################################################
# COMPUTE

for i in range(n):
p=pop0(p,br,dr)
pop.append(p)

##################################################
# SHOW RESULTS

for i in range(n+1):
print(‘Year %5d = Citizens. %8d \n’ %(baseYear+i, pop[i]) )

x = np.linspace(1,len(pop),len(pop))

plt.plot(x, pop, ‘bo’)
plt.show()

#####################################################
# STORE VALUES ON DISK
#
# For this see the online article
# https://www.pythonforthelab.com/blog/introduction-to-storing-data-in-files/
#
# Saves the plot data automatically in a file with a header and two columns

data = np.column_stack((x,pop))
ts = time.gmtime()
t=time.strftime(“%c”, ts) # format time data into ISO format
t=t.replace(‘ ‘,’-‘)
t=t.replace(‘:’,’-‘)
header=’br=’+str(br)+’,’+’dr=’+str(dr)+’\n’+’X-Column, Y-Column’
fname=’hxyPTL’+t+’br=’+str(br)+’-‘+’dr=’+str(dr)
np.savetxt(fname+’.txt’, data,header=header)

###########################################
# Compute Change of POP

n1=pop[0]
n2=pop[len(pop)-1]
Increase=(n2-n1)/(n1/100)

print(“From Year %5d, until Year %5d, a change of %2.2f percent \n” % (baseYear, baseYear+n,Increase) )

plt.close()

####################################################
# REAL DATA
#
# UN Demographic Yearbook 2017
# https://unstats.un.org/unsd/demographic-social/products/dyb/dybsets/2017.pdf
#
# Basic Tables UN
# https://unstats.un.org/unsd/demographic-social/products/vitstats/seratab1.pdf
#
# UN public tables
# http://data.un.org/Explorer.aspx?d=POP
#
# UN Rate of population change
# http://data.un.org/Data.aspx?d=PopDiv&f=variableID%3a47
# https://www.un.org/en/development/desa/population/index.asp
”’
Population number ? 6958169

Birthrate in % ? 1.9

Deathrate in % ? 0.77

How many cycles ? 15

What is your Base Year ? 2010
Year 2010 = Citizens. 6958169

Year 2011 = Citizens. 7036796

Year 2012 = Citizens. 7116312

Year 2013 = Citizens. 7196726

Year 2014 = Citizens. 7278049

Year 2015 = Citizens. 7360291

Year 2016 = Citizens. 7443462

Year 2017 = Citizens. 7527573

Year 2018 = Citizens. 7612635

Year 2019 = Citizens. 7698658

Year 2020 = Citizens. 7785653

Year 2021 = Citizens. 7873630

Year 2022 = Citizens. 7962602

Year 2023 = Citizens. 8052580

Year 2024 = Citizens. 8143574

Year 2025 = Citizens. 8235596

From Year 2010, until Year 2025, a change of 18.36 percent

Real UN data for 2010 – 2015
2010 2011 2012 2013 2014 2015
6 958 169 7 043 009 7 128 177 7 213 426 7 298 453 7.383.009

########################################################
# STORING DATA
#
# https://www.pythonforthelab.com/blog/introduction-to-storing-data-in-files/
#
# Example of saved file:
file name:
hxyPTLSun-Apr–7-08-55-46-2019br=0.019-dr=0.0077.txt

# br=0.019,dr=0.0077
# X-Column, Y-Column
1.000000000000000000e+00 6.958169000000000000e+06
2.000000000000000000e+00 7.036796309700000100e+06
3.000000000000000000e+00 7.116312107999609783e+06
4.000000000000000000e+00 7.196726434820005670e+06
5.000000000000000000e+00 7.278049443533471785e+06
6.000000000000000000e+00 7.360291402245399542e+06
7.000000000000000000e+00 7.443462695090772584e+06
8.000000000000000000e+00 7.527573823545298539e+06
9.000000000000000000e+00 7.612635407751359977e+06
1.000000000000000000e+01 7.698658187858950347e+06
1.100000000000000000e+01 7.785653025381756946e+06
1.200000000000000000e+01 7.873630904568570666e+06
1.300000000000000000e+01 7.962602933790194802e+06
1.400000000000000000e+01 8.052580346942024305e+06
1.500000000000000000e+01 8.143574504862469621e+06
1.600000000000000000e+01 8.235596896767416038e+06
”’