Magic Commands

Posted on by

Magic Command: %sh

%sh เป็นการใช้คำสั่ง shell

%sh ps
%sh ps | grep 'java'

การอ้างถึงพาทต่างๆของ %sh เป็นพาทของ shell

%sh ls -l /

ต่างกับ %fs ที่การอ้างพาท เป็นพาทของ dbfs 

%fs ls /

Databricks File System – DBFS

  • DBFS is a layer over a cloud-based object store
  • Files in DBFS are persisted to the object store
  • The lifetime of files in the DBFS are NOT tied to the lifetime of our cluster

Magic Command: Other Languages

%python

print("Hello Python!")
%scala

println("Hello Scala!")
%sql

select "Hello SQL!"
%r

print("Hello R!", quote=FALSE)

Magic Command: %md

%md
%md
### Magic Command: %md

Our favorite Magic Command **%md** allows us to render Markdown in a cell:
* Double click this cell to begin editing it
* Then hit `Esc` to stop editing

# Title One
## Title Two
### Title Three

This is a test of the emergency broadcast system. This is only a test.

This is text with a **bold** word in it.

This is text with an *italicized* word in it.

This is an ordered list
0. once
0. two
0. three

This is an unordered list
* apples
* peaches
* bananas

Links/Embedded HTML: <a href="http://bfy.tw/19zq" target="_blank">What is Markdown?</a>

Images:
![Spark Engines](https://files.training.databricks.com/images/Apache-Spark-Logo_TM_200px.png)

And of course, tables:

| Name  | Age | Sex    |
|-------|-----|--------|
| Tom   | 32  | Male   |
| Mary  | 29  | Female |
| Dick  | 73  | Male   |
| Sally | 55  | Female |

Magic Command: %run

  • You can run a notebook from another notebook by using the Magic Command %run
  • All variables & functions defined in that other notebook will become available in your current notebook
%run "./Includes/Classroom-Setup"

Magic Command: %fs

It is a wrapper around dbutils.fs and it is the Magic Command known as %fs.

The following call is equivalent to the display( dbutils.fs.ls("/mnt/training") ) – there is no real difference between the two.

%fs ls /mnt/training
%fs head /mnt/training/pageviews_by_second.tsv