Auto increment id
- apache spark – Auto increment id in delta table while inserting – Stack Overflow
- CREATE TABLE [USING] (Databricks SQL) | Databricks on AWS
Databricks has IDENTITY columns for hosted Spark
[ GENERATED ALWAYS AS ( expr ) |
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY
[ ( [ START WITH start ] [ INCREMENT BY step ] ) ]
]
This works on Delta tables. Example:
create table gen1 (
id long GENERATED ALWAYS AS IDENTITY
, t string
)
Requires Runtime version 10.4 or above.
hash function (Databricks SQL)