You have to use coalesce or NOT NULL to substitute null values in select query.

Check below if it helps:

Try this:

spark.sql("""
select
 patient_id,
 CASE 
 when cough is NOT NULL THEN cough
 else 0
 END as cough,
 CASE 
 when feaver is NOT NULL THEN feaver
 else 0
 END as feaver,
 CASE 
 when `head ache` is NOT NULL THEN `head ache`
 else 0
 END as `head ache`
 from ( 
select * from patient
)
PIVOT(
  Count(dx)
  for dx in ('cough','feaver','head ache')
)
;
""").show()

The output will be:

if you want it to be dynamic

dist=spark.sql("select collect_set(dx) from patient;").toPandas()
val=spark.sql("""
select
 patient_id,
 coalesce(cough,0) as `cough`,
 coalesce(feaver,0) as `feaver`,
 coalesce(`head ache`,0) as `head ache`
 from ( 
select * from patient
)
PIVOT(
  Count(dx)
  for dx in """
+
str(tuple(map(tuple, *dist.values))[0])
+
"""
)
;
""")

Coalesce is supported in Databricks tsql

%sql

Select
   f.global_key as glb_num
  ,f.global_name as glb_name
  ,coalesce(r.tm2016, 0) as tm2016
  ,coalesce,(r.tm2017, 0) as tm2017
  ,coalesce,(r.tm2018, 0) as tm2018

FROM co_family as f -- Company Structure
Left Join main_revenue as r -- Revenue
  On f.global_key = r.global_key

You can use NULLIF() function and replace the column having empty string with null value.

select nullif(<column_name>,'') from <table-name>

Nullif returns null if the 1st expression in it equals to the 2nd expression. Here, if column has empty string, then nullif returns null.