Magic Commands

Magic Command: %sh

%sh เป็นการใช้คำสั่ง shell

%sh ps
%sh ps | grep 'java'

การอ้างถึงพาทต่างๆของ %sh เป็นพาทของ shell

%sh ls -l /

ต่างกับ %fs ที่การอ้างพาท เป็นพาทของ dbfs

%fs ls /

Databricks File System – DBFS

  • DBFS is a layer over a cloud-based object store
  • Files in DBFS are persisted to the object store
  • The lifetime of files in the DBFS are NOT tied to the lifetime of our cluster

Magic Command: Other Languages


print("Hello Python!")

println("Hello Scala!")

select "Hello SQL!"

print("Hello R!", quote=FALSE)

Magic Command: %md

### Magic Command: %md

Our favorite Magic Command **%md** allows us to render Markdown in a cell:
* Double click this cell to begin editing it
* Then hit `Esc` to stop editing

# Title One
## Title Two
### Title Three

This is a test of the emergency broadcast system. This is only a test.

This is text with a **bold** word in it.

This is text with an *italicized* word in it.

This is an ordered list
0. once
0. two
0. three

This is an unordered list
* apples
* peaches
* bananas

Links/Embedded HTML: <a href="" target="_blank">What is Markdown?</a>

![Spark Engines](

And of course, tables:

| Name  | Age | Sex    |
| Tom   | 32  | Male   |
| Mary  | 29  | Female |
| Dick  | 73  | Male   |
| Sally | 55  | Female |

Magic Command: %run

  • You can run a notebook from another notebook by using the Magic Command %run
  • All variables & functions defined in that other notebook will become available in your current notebook
%run "./Includes/Classroom-Setup"

Magic Command: %fs

It is a wrapper around dbutils.fs and it is the Magic Command known as %fs.

The following call is equivalent to the display("/mnt/training") ) – there is no real difference between the two.

%fs ls /mnt/training
%fs head /mnt/training/pageviews_by_second.tsv