PyYAML

ติดตั้ง PyYAML

python -m pip install pyyaml

Example1

import yaml

dict1 = yaml.load("""
    name: Vorlin Laruknuzum
    sex: Male
    class: Priest
    title: Acolyte
    hp: [32, 71]
    sp: [1, 13]
    gold: 423
    inventory:
    - a Holy Book of Prayers (Words of Wisdom)
    - an Azure Potion of Cure Light Wounds
    - a Silver Wand of Wonder
    """)

print(type(dict1))
print(dict1)

ผลการรัน จะมี Warning ว่า deprecated

YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
  """)

<class 'dict'>
{'name': 'Vorlin Laruknuzum', 'sex': 'Male', 'class': 'Priest', 'title': 'Acolyte', 'hp': [32, 71], 'sp': [1, 13], 'gold': 423, 'inventory': ['a Holy Book of Prayers (Words of Wisdom)', 'an Azure Potion of Cure Light Wounds', 'a Silver Wand of Wonder']}

นำผลลัพธ์มาจัด format จะได้ประมาณนี้

{
    "name": "Vorlin Laruknuzum",
    "sex": "Male",
    "class": "Priest",
    "title": "Acolyte",
    "hp": [
        32,
        71
    ],
    "sp": [
        1,
        13
    ],
    "gold": 423,
    "inventory": [
        "a Holy Book of Prayers (Words of Wisdom)",
        "an Azure Potion of Cure Light Wounds",
        "a Silver Wand of Wonder"
    ]
}

Example2

import yaml

st1 = yaml.dump({'name': "The Cloak 'Colluin'", 'depth': 5, 'rarity': 45,'weight': 10, 'cost': 50000, 'flags': ['INT', 'WIS', 'SPEED', 'STEALTH']})
print(type(st1))
print(st1)

ผลการรัน

<class 'str'>
cost: 50000
depth: 5
flags:
- INT
- WIS
- SPEED
- STEALTH
name: The Cloak 'Colluin'
rarity: 45
weight: 10

Loading YAML

The function yaml.load converts a YAML document to a Python object.

import yaml

li1 = yaml.load("""
    - Hesperiidae
    - Papilionidae
    - Apatelodidae
    - Epiplemidae
    """)
print(type(li1))
print(li1)
YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
  """)

<class 'list'>
['Hesperiidae', 'Papilionidae', 'Apatelodidae', 'Epiplemidae']

yaml.load accepts a byte string, a Unicode string, an open binary file object, or an open text file object. A byte string or a file must be encoded with utf-8utf-16-be or utf-16-le encoding. yaml.load detects the encoding by checking the BOM (byte order mark) sequence at the beginning of the string/file. If no BOM is present, the utf-8 encoding is assumed.

import yaml

dict1 = yaml.load("""
    hello: สวัสดี
    """)
print(type(dict1))
print(dict1)
<class 'dict'>
{'hello': 'สวัสดี'}

แต่ถ้าอ่านจากไฟล์ ภาษาไทยอ่านไม่ออก

import yaml

stream = open('document.yaml', 'r')
dict1 = yaml.load(stream)

print(type(dict1))
print(dict1)
<class 'dict'>
{'hello': 'สวัสดี'}

ใช้ safe_load() จะไม่มี Warning ว่า deprecated

import yaml

stream = open('document.yaml', 'r')
dict1 = yaml.safe_load(stream)

print(type(dict1))
print(dict1)

ติดตั้ง Anaconda3 บน Windows10

Release notes

บน Windows จะติดตั้งอยู่ทีพาท

C:\Users\jack\Anaconda3

ให้เซ็ต Environment Variables | System variables ตัวแปร Path

C:\Users\jack\Anaconda3
C:\Users\jack\Anaconda3\Scripts
C:\Users\jack\Anaconda3\Library\bin

กรณีติดตั้ง Anaconda 2020.02

ตรวจสอบการติดตั้ง

> python --version
Python 3.7.6
> python -m pip --version
pip 20.0.2 from C:\Users\jack\Anaconda3\lib\site-packages\pip (python 3.7)
> conda --version
conda 4.8.2

ดูรายการแพจเกจ

> conda list
> pip list

ตรวจสอบโมดูล ssl

> python -m ssl
Traceback (most recent call last):
  File "c:\Users\jack\Anaconda3\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "c:\Users\jack\Anaconda3\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "c:\Users\jack\Anaconda3\lib\ssl.py", line 98, in <module>
    import _ssl             # if we can't import it, let the error propagate
ImportError: DLL load failed: The specified module could not be found.

ถ้าขึ้น error แบบนี้ให้กลับไปตรวจสอบ %PATH% ว่าชี้ไปที่ C:\Users\jack\Anaconda3\Library\bin แล้วรึยัง

แต่ถ้าไม่มีอะไรขึ้นแปลว่าใช้ได้

อัพเดท pip

> pip install --upgrade pip

ดูเวอร์ชัน pip

> python -m pip --version
pip 21.2.4 from C:\Users\phaisarn_su\Anaconda3\lib\site-packages\pip (python 3.7)

ติดตั้ง หรือ อัพเดท PySpark

> conda install pyspark

Python3.7 ติดต่อดาต้าเบสผ่าน ODBC ด้วย pyodbc

ติดตั้ง curl

sudo apt install curl

ติดตั้ง Microsoft ODBC driver for SQL Server

ติดตั้ง Microsoft ODBC 17

sudo su
curl https://packages.microsoft.com/keys/microsoft.asc | apt-key add -

#Ubuntu 18.04
curl https://packages.microsoft.com/config/ubuntu/18.04/prod.list > /etc/apt/sources.list.d/mssql-release.list

exit

sudo apt update
sudo ACCEPT_EULA=Y apt-get install -y msodbcsql17

# optional: for bcp and sqlcmd
sudo ACCEPT_EULA=Y apt-get install -y mssql-tools
echo 'export PATH="$PATH:/opt/mssql-tools/bin"' >> ~/.bashrc
source ~/.bashrc
# optional: for unixODBC development headers
sudo apt install -y unixodbc-dev

ติดตั้ง pyodbc

python3 -m pip install pyodbc

ทดลอง import pyodbc ถ้าไม่มี error ก็แสดงว่าติดตั้งได้ละ

$ python3
Python 3.7.5 (default, Feb 23 2021, 13:22:40)
[GCC 8.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pyodbc
>>>

Connet ไป SQL Server

# test.py

import pyodbc 
# Some other example server values are
# server = 'localhost\sqlexpress' # for a named instance
# server = 'myserver,port' # to specify an alternate port
server = 'tcp:myserver.database.windows.net' 
database = 'mydb' 
username = 'myusername' 
password = 'mypassword' 
cnxn = pyodbc.connect('DRIVER={ODBC Driver 17 for SQL Server};SERVER='+server+';DATABASE='+database+';UID='+username+';PWD='+ password)
cursor = cnxn.cursor()

ลองรัน ถ้าไม่มี error ก็ connect ได้ละ

$ python3 test.py

Run query

#Sample select query
cursor.execute("SELECT @@version;") 
row = cursor.fetchone() 
while row: 
    print(row[0])
    row = cursor.fetchone()

ผลการรัน

$ python3 test.py
Microsoft SQL Server 2019 (RTM) - 15.0.2000.5 (X64)
        Sep 24 2019 13:48:23
        Copyright (C) 2019 Microsoft Corporation
        Developer Edition (64-bit) on Windows 10 Pro 10.0 <X64> (Build 19042: ) (Hypervisor)

Insert a row

#Sample insert query
count = cursor.execute("""
INSERT INTO SalesLT.Product (Name, ProductNumber, StandardCost, ListPrice, SellStartDate) 
VALUES (?,?,?,?,?)""",
'SQL Server Express New 20', 'SQLEXPRESS New 20', 0, 0, CURRENT_TIMESTAMP).rowcount
cnxn.commit()
print('Rows inserted: ' + str(count))

ปิด connection

cnxn.close()

ติดตั้ง Python3.7 บน Ubuntu 18.04

ติดตั้ง Python3.7

อัพเดท

sudo apt update

ตรวจสอบเวอร์ชันปัจจุบัน

$ python3 --version
Python 3.6.9

ติดตั้ง Python3.7

sudo apt install python3.7

ตรวจสอบการติดตั้ง

$ python3.7 --version
Python 3.7.5

กำหนด default python ให้เป็น Python3.7

sudo rm /usr/bin/python3
sudo ln -s /usr/bin/python3.7 /usr/bin/python3

ตรวจสอบเวอร์ชัน

$ python3 --version
Python 3.7.5

ปัญหาเมื่อคำสั่ง python3 เป็นเวอร์ชัน 3.7

พอคำสั่ง python3 เป็นเวอร์ชัน 3.7 รัน apt update แล้ว error เลย

$ sudo apt update
Hit:1 http://th.archive.ubuntu.com/ubuntu bionic InRelease
Hit:2 http://th.archive.ubuntu.com/ubuntu bionic-updates InRelease
Hit:3 http://th.archive.ubuntu.com/ubuntu bionic-backports InRelease
Hit:4 http://security.ubuntu.com/ubuntu bionic-security InRelease
Traceback (most recent call last):
  File "/usr/lib/cnf-update-db", line 8, in <module>
    from CommandNotFound.db.creator import DbCreator
  File "/usr/lib/python3/dist-packages/CommandNotFound/db/creator.py", line 11, in <module>
    import apt_pkg
ModuleNotFoundError: No module named 'apt_pkg'
Reading package lists... Done
E: Problem executing scripts APT::Update::Post-Invoke-Success 'if /usr/bin/test -w /var/lib/command-not-found/ -a -e /usr/lib/cnf-update-db; then /usr/lib/cnf-update-db > /dev/null; fi'
E: Sub-process returned an error code

วิธีแก้ไข

sudo apt remove  python3-apt
sudo apt install python3-apt

รัน apt update อีกทีจะใช้ได้ปกติละ

$ sudo apt update
Hit:1 http://th.archive.ubuntu.com/ubuntu bionic InRelease
Hit:2 http://th.archive.ubuntu.com/ubuntu bionic-updates InRelease
Hit:3 http://th.archive.ubuntu.com/ubuntu bionic-backports InRelease
Hit:4 http://security.ubuntu.com/ubuntu bionic-security InRelease
Reading package lists... Done
Building dependency tree
Reading state information... Done
All packages are up to date.

ติดตั้ง pip3

sudo apt install python3-pip

ตรวจสอบเวอร์ชัน

$ pip3 --version
pip 9.0.1 from /usr/lib/python3/dist-packages (python 3.7)

อัพเดท pip3

python3 -m pip install --upgrade pip

ตรวจสอบเวอร์ชัน

$ pip3 --version
WARNING: pip is being invoked by an old script wrapper. This will fail in a future version of pip.
Please see https://github.com/pypa/pip/issues/5599 for advice on fixing the underlying issue.
To avoid this problem you can invoke Python with '-m pip' instead of running pip directly.
pip 21.2.4 from /home/jack/.local/lib/python3.7/site-packages/pip (python 3.7)
$ python3 -m pip --version
pip 21.2.4 from /home/jack/.local/lib/python3.7/site-packages/pip (python 3.7)

To list all the installed pip packages:

python3 -m pip list

To upgrade an installed package to the latest version:

python3 -m pip install --upgrade package_name

To uninstall a package run:

python3 -m pip uninstall package_name

ติดตั้ง python3.7-dev

ติดตั้ง python3-dev แบบระบุเวอร์ชันเป็น 3.7

sudo apt install python3.7-dev

การใช้ argparse ใน Python3

  • argparse — Parser for command-line options, arguments and sub-commands

Example

The following code is a Python program that takes a list of integers and produces either the sum or the max:

import argparse

parser = argparse.ArgumentParser(description='Process some integers.')
parser.add_argument('integers', metavar='N', type=int, nargs='+',
                    help='an integer for the accumulator')
parser.add_argument('--sum', dest='accumulate', action='store_const',
                    const=sum, default=max,
                    help='sum the integers (default: find the max)')

args = parser.parse_args()
print(args.accumulate(args.integers))

Assuming the Python code above is saved into a file called prog.py, it can be run at the command line and provides useful help messages:

$ python3 prog.py -h
usage: prog.py [-h] [--sum] N [N ...]

Process some integers.

positional arguments:
  N           an integer for the accumulator

optional arguments:
  -h, --help  show this help message and exit
  --sum       sum the integers (default: find the max)

When run with the appropriate arguments, it prints either the sum or the max of the command-line integers:

$ python3 prog.py 1 5 3 2 4
5
$ python3 prog.py 1 5 3 2 4 --sum
15

If invalid arguments are passed in, it will issue an error:

$ python3 prog.py a b c
usage: prog.py [-h] [--sum] N [N ...]
prog.py: error: argument N: invalid int value: 'a'
$ python3 prog.py
usage: prog.py [-h] [--sum] N [N ...]
prog.py: error: the following arguments are required: N

Example2

#!/usr/bin/python3

import argparse

if __name__ == '__main__':
        parser = argparse.ArgumentParser()
        parser.add_argument('folder', help='Path to the folder to scan')
        parser.add_argument('output', help='Path of the output file without the extension')
        parser.add_argument('-f', '--format', help='Format of the output', default='pdf', \
                choices=['bmp', 'gif', 'jpg', 'png', 'pdf', 'svg'])
        parser.add_argument('-v', '--view', action='store_true', help='View the graph')
        parser.add_argument('-c', '--cluster', action='store_true', help='Create a cluster for each subfolder')
        parser.add_argument('--cluster-labels', dest='cluster_labels', action='store_true', help='Label subfolder clusters')
        parser.add_argument('-s', '--strict', action='store_true', help='Rendering should merge multi-edges', default=False)
        args = parser.parse_args()
        print(args.folder)
        print(args.cluster)
        print(args.cluster_labels)
        print(args.strict)
        print(args.format)
        print(args.output)
$ ./prog.py ip_folder file_out -c -f png
ip_folder
True
False
False
png
file_out

ดู help

$ ./prog.py -h
usage: prog.py [-h] [-f {bmp,gif,jpg,png,pdf,svg}] [-v] [-c]
               [--cluster-labels] [-s]
               folder output

positional arguments:
  folder                Path to the folder to scan
  output                Path of the output file without the extension

optional arguments:
  -h, --help            show this help message and exit
  -f {bmp,gif,jpg,png,pdf,svg}, --format {bmp,gif,jpg,png,pdf,svg}
                        Format of the output
  -v, --view            View the graph
  -c, --cluster         Create a cluster for each subfolder
  --cluster-labels      Label subfolder clusters
  -s, --strict          Rendering should merge multi-edges

ไม่ใส่ค่าให้ แล้ว error

$ ./prog.py
usage: prog.py [-h] [-f {bmp,gif,jpg,png,pdf,svg}] [-v] [-c]
               [--cluster-labels] [-s]
               folder output
prog.py: error: the following arguments are required: folder, output

การใช้ logging ใน Python3

A simple example

#!/usr/bin/python3

import logging
logging.warning('Watch out!')  # will print a message to the console
logging.info('I told you so')  # will not print anything

ผลการรัน

WARNING:root:Watch out!

Logging to a file

import logging
logging.basicConfig(filename='example.log',level=logging.DEBUG)
logging.debug('This message should go to the log file')
logging.info('So should this')
logging.warning('And this, too')

ผลการรันในไฟล์ example.log

DEBUG:root:This message should go to the log file
INFO:root:So should this
WARNING:root:And this, too

This example also shows how you can set the logging level which acts as the threshold for tracking. In this case, because we set the threshold to DEBUG, all of the messages were printed.

Logging from multiple modules

# myapp.py
import logging
import mylib

def main():
    logging.basicConfig(filename='myapp.log', level=logging.INFO)
    logging.info('Started')
    mylib.do_something()
    logging.info('Finished')

if __name__ == '__main__':
    main()
# mylib.py
import logging

def do_something():
    logging.info('Doing something')

ผลการรันในไฟล์  myapp.log:

INFO:root:Started
INFO:root:Doing something
INFO:root:Finished

Logging variable data

To log variable data, use a format string for the event description message and append the variable data as arguments.

import logging
logging.warning('%s before you %s', 'Look', 'leap!')

ผลการรัน

WARNING:root:Look before you leap!

Changing the format of displayed messages

import logging
logging.basicConfig(format='%(levelname)s:%(message)s', level=logging.DEBUG)
logging.debug('This message should appear on the console')
logging.info('So should this')
logging.warning('And this, too')

ผลการรัน

DEBUG:This message should appear on the console
INFO:So should this
WARNING:And this, too

Displaying the date/time in messages

import logging
logging.basicConfig(format='%(asctime)s %(message)s')
logging.warning('is when this event was logged.')

ผลการรัน

2021-09-15 13:37:34,794 is when this event was logged.

The default format for date/time display (shown above) is like ISO8601 or RFC 3339. If you need more control over the formatting of the date/time, provide a datefmt argument to basicConfig, as in this example:

import logging
logging.basicConfig(format='%(asctime)s %(message)s', datefmt='%m/%d/%Y %I:%M:%S %p')
logging.warning('is when this event was logged.')

ผลการรัน

09/15/2021 01:38:42 PM is when this event was logged.

The format of the datefmt argument is the same as supported by time.strftime().

Configuring Logging

Creating loggers, handlers, and formatters explicitly using Python code that calls the configuration methods listed above.

import logging

# create logger
logger = logging.getLogger('simple_example')
logger.setLevel(logging.DEBUG)

# create console handler and set level to debug
ch = logging.StreamHandler()
ch.setLevel(logging.DEBUG)

# create formatter
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')

# add formatter to ch
ch.setFormatter(formatter)

# add ch to logger
logger.addHandler(ch)

# 'application' code
logger.debug('debug message')
logger.info('info message')
logger.warning('warn message')
logger.error('error message')
logger.critical('critical message')

ผลการรัน

2021-09-15 13:45:40,226 - simple_example - DEBUG - debug message
2021-09-15 13:45:40,226 - simple_example - INFO - info message
2021-09-15 13:45:40,226 - simple_example - WARNING - warn message
2021-09-15 13:45:40,226 - simple_example - ERROR - error message
2021-09-15 13:45:40,226 - simple_example - CRITICAL - critical message

ทำไฟล์ config แยก

import logging
import logging.config

logging.config.fileConfig('logging.conf')

# create logger
logger = logging.getLogger('simpleExample')

# 'application' code
logger.debug('debug message')
logger.info('info message')
logger.warning('warn message')
logger.error('error message')
logger.critical('critical message')

ไฟล์ logging.conf

[loggers]
keys=root,simpleExample

[handlers]
keys=consoleHandler

[formatters]
keys=simpleFormatter

[logger_root]
level=DEBUG
handlers=consoleHandler

[logger_simpleExample]
level=DEBUG
handlers=consoleHandler
qualname=simpleExample
propagate=0

[handler_consoleHandler]
class=StreamHandler
level=DEBUG
formatter=simpleFormatter
args=(sys.stdout,)

[formatter_simpleFormatter]
format=%(asctime)s - %(name)s - %(levelname)s - %(message)s
datefmt=

ผลการรัน

2021-09-15 13:51:47,993 - simpleExample - DEBUG - debug message
2021-09-15 13:51:47,993 - simpleExample - INFO - info message
2021-09-15 13:51:47,993 - simpleExample - WARNING - warn message
2021-09-15 13:51:47,993 - simpleExample - ERROR - error message
2021-09-15 13:51:47,993 - simpleExample - CRITICAL - critical message

Logging Levels

LevelNumeric value
CRITICAL50
ERROR40
WARNING30
INFO20
DEBUG10
NOTSET0

Using Handlers

ตัวอย่าง console เซ็ตเป็น logging.WARNING ส่วน file เซ็ตเป็น logging.ERROR

# logging_example.py

import logging

# Create a custom logger
logger = logging.getLogger(__name__)

# Create handlers
c_handler = logging.StreamHandler()
f_handler = logging.FileHandler('file.log')
c_handler.setLevel(logging.WARNING)
f_handler.setLevel(logging.ERROR)

# Create formatters and add it to handlers
c_format = logging.Formatter('%(name)s - %(levelname)s - %(message)s')
f_format = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
c_handler.setFormatter(c_format)
f_handler.setFormatter(f_format)

# Add handlers to the logger
logger.addHandler(c_handler)
logger.addHandler(f_handler)

logger.warning('This is a warning')
logger.error('This is an error')

ผลการรันที่ console

__main__ - WARNING - This is a warning
__main__ - ERROR - This is an error

ผลการรันที่ไฟล์ file.log

2021-09-15 14:04:23,318 - __main__ - ERROR - This is an error

Capturing Stack Traces

import logging

a = 5
b = 0

try:
  c = a / b
except Exception as e:
  logging.error("Exception occurred", exc_info=True)

หรือ

import logging

a = 5
b = 0
try:
  c = a / b
except Exception as e:
  logging.exception("Exception occurred")

ผลการรัน

ERROR:root:Exception occurred
Traceback (most recent call last):
  File "./helloworld.py", line 7, in <module>
    c = a / b
ZeroDivisionError: division by zero

Configuring Logging in a YAML format

ติดตั้ง PyYAML

python3 -m pip install pyyaml

ไฟล์ config.yaml

version: 1
formatters:
  simple:
    format: '%(asctime)s - %(name)s - %(levelname)s - %(message)s'
handlers:
  console:
    class: logging.StreamHandler
    level: DEBUG
    formatter: simple
    stream: ext://sys.stdout
loggers:
  sampleLogger:
    level: DEBUG
    handlers: [console]
    propagate: no
root:
  level: DEBUG
  handlers: [console]
import logging
import logging.config
import yaml

with open('config.yaml', 'r') as f:
    config = yaml.safe_load(f.read())
    logging.config.dictConfig(config)

logger = logging.getLogger(__name__)

logger.debug('This is a debug message')

ผลการรัน

2021-09-15 14:30:42,270 - __main__ - DEBUG - This is a debug message