You may try the following which uses explode to split the list of Employee Ids into different rows before joining them and using collect_list to aggregate the entries into a list.

Using spark sql:

NB. Ensure Employees is available as a table/view eg EmployeeData.createOrReplaceTempView("Employees")

WITH dept_employees AS (
    SELECT
        DeptId,
        DeptName,
        explode(EmployeeIDs)
    FROM
        vw_TestView
)
SELECT
    d.DeptId,
    d.DeptName,
    collect_list(e.EmpID) as EmployeeIDs,
    collect_list(e.EmpName) as EmployeeNames
FROM
    dept_employees d
INNER JOIN
    Employees e ON d.col=e.EmpID
GROUP BY
    d.Deptid,
    d.DeptName

or using pyspark api:

from pyspark.sql import functions as F

output_df = (
    departMentData.select(
        F.col("DeptId"),
        F.col("DeptName"),
        F.explode("EmployeeIDs")
    )
    .alias("d")
    .join(
        EmployeeData.alias("e"),
        F.col("d.col")==F.col("e.EmpID"),
        "inner"
    )
    .groupBy("d.DeptId","d.DeptName")
    .agg(
        F.collect_list("e.EmpID").alias("EmployeeIDs"),
        F.collect_list("e.EmpName").alias("EmployeeNames")
    )
)

Let me know if this works for you.

Answer from ggordon on Stack Overflow
🌐
Databricks
docs.databricks.com › sql language reference › functions › built-in functions › alphabetical list of built-in functions › array_join function
array_join function | Databricks on AWS
September 11, 2026 - > SELECT array_join(array('hello', 'world'), ','); hello,world > SELECT array_join(array('hello', NULL ,'world'), ','); hello,world > SELECT array_join(array('hello', NULL ,'world'), ',', '*'); hello,*,world
🌐
Databricks
api-docs.databricks.com › python › pyspark › latest › pyspark.sql › api › pyspark.sql.functions.array_join.html
pyspark.sql.functions.array_join — PySpark master documentation
Concatenates the elements of column using the delimiter. Null values are replaced with null_replacement if set, otherwise they are ignored · pyspark.sql.functions.slice
🌐
Microsoft Learn
learn.microsoft.com › en-us › azure › databricks › pyspark › reference › functions › array_join
array_join - Azure Databricks | Microsoft Learn
June 4, 2026 - array\_join function in PySpark: Returns a string column by concatenating the elements of the input array column using the delimiter.
🌐
Databricks
docs.databricks.com › pyspark reference › functions › array_join
array_join | Databricks on AWS
June 4, 2026 - array\_join function in PySpark: Returns a string column by concatenating the elements of the input array column using the delimiter.
🌐
Stack Overflow
stackoverflow.com › questions › 77586299 › databricks-sql-array-join
apache spark sql - Databricks SQL array join - Stack Overflow
I have created query in databricks sql thats grouping text values by certain key. The problem is that in some cases different date in grouping key value creates 2 separate rows ... Final AS ( SELECT ,1.PRIMARY_KEY ,1.DUMMYCOLUMN ,1.DATE_START ,1.DATE_END ,ARRAY_JOIN(COLLECT_SET(Sales), ', ') AS TEST FROM Contract GROUP BY PRIMARY_KEY, DATE_START, DATE_END) SELECT * FROM FINAL
🌐
Databricks
databricks-prod-cloudfront.cloud.databricks.com › public › 4027ec902e239c93eaaa8714f173bcfc › 142158605138935 › 3773509768457258 › 7497868276316206 › latest.html
Examples of New Built-in Functions and Higher-order Functions - Databricks
%md #### array_join(array<String>, String[, String]): String Concatenates the elements of the given array using the delimiter and an optional string to replace nulls.
Find elsewhere
🌐
Databricks Documentation
docs.databricks.com › pyspark reference › functions › array_join
array_join | Databricks on Google Cloud
array\_join function in PySpark: Returns a string column by concatenating the elements of the input array column using the delimiter.
🌐
Databricks
docs.databricks.com › aws › en › notebooks › source › apache-spark-functions.html
apache-spark-2.4-functions - Databricks
%md #### array_join(array<String>, String[, String]): String Concatenates the elements of the given array using the delimiter and an optional string to replace nulls.
🌐
Databricks Community
community.databricks.com › databricks community › data engineering › is it possible to control the ordering of the array values created by array_agg()?
Is it possible to control the ordering of the array values created by array_agg()?
April 10, 2024 - with tmp as ( select * from (values (1, 1, 'cherry'), (1, 3, 'strawberries'), (1, 2, 'acerola'), (1, 5, 'banan'), (1, 4, 'lemon') ) as tab(custid, hist, alf) order by custid asc, hist asc ) select custid, array_join(array_agg(alf), ' -> ') as move from tmp group by custid order by custid asc;
🌐
Databricks
databricks-prod-cloudfront.cloud.databricks.com › public › 4027ec902e239c93eaaa8714f173bcfc › 6907863442366616 › 2357259023612622 › 1911168149879216 › latest.html
8. Joins - Databricks
import org.apache.spark.sql.functions.expr person.withColumnRenamed("id", "personId") .join(sparkStatus, expr("array_contains(spark_status, id)")).show()
🌐
Microsoft Learn
learn.microsoft.com › en-us › azure › databricks › sql › language-manual › functions › concat
concat function - Azure Databricks - Databricks SQL | Microsoft Learn
June 24, 2026 - When concatenating arrays, Azure Databricks raises COLLECTION_SIZE_LIMIT_EXCEEDED if the result exceeds the array size limit. COLLECTION_SIZE_LIMIT_EXCEEDED · > SELECT concat('Spark', 'SQL'); SparkSQL > SELECT concat(array(1, 2, 3), array(4, 5), array(6)); [1,2,3,4,5,6] || (pipe pipe sign) operator · array_join function ·
🌐
Databricks Community
community.databricks.com › databricks community › data engineering › use array in where in clause
Solved: Use Array in WHERE IN clause - Databricks Community - 129326
August 25, 2025 - In your SQL statement, where IDs in (ID) is comparing a string column (IDs) with an array variable (ARRAY<STRING>). Databricks SQL does not allow IN to directly take an array — IN expects a list of scalar values, not an array object.
🌐
Databricks Community
community.databricks.com › t5 › data-engineering › error-while-using-array-contains-function-in-left-join-condition › td-p › 7441
Error while using Array_contains function in left join condition
March 27, 2023 - 'Item_id' is column in array format like ["ba1b-5fbe1547ddd5", "88f9-ac3b93334f69", "8bba-4075a47eb814"] in table1 and table2 has column Id with single value like ba1b-5fbe1547ddd5. While join two table select table1.*, table2.* from table1 left join table2 on array_contains(table2.id, table1.ite...