You need to use a union.

Off the top of my head:

all_settings="{{ foo|map(attribute='settings')|union(bar|map(attribute='settings')) }}"
Answer from Michael Hampton on serverfault.com
🌐
Ansible
docs.ansible.com › projects › ansible › latest › collections › community › general › lists_mergeby_filter.html
community.general.lists_mergeby filter – Merge two or more lists of dictionaries by a given attribute — Ansible Community Documentation
Merge three lists ansible.builtin.debug: var: r vars: list1: - {index: a, value: 123} - {index: b, value: 4} list2: - {index: a, foo: bar} - {index: c, foo: baz} list3: - {index: d, foo: qux} r: "{{ [list1, list2, list3] | community.general.lists_mergeby('index') }}" # r: # - {index: a, foo: bar, value: 123} # - {index: b, value: 4} # - {index: c, foo: baz} # - {index: d, foo: qux} - name: Example 3.
Discussions

Combine Lists of Objects in Ansible - DevOps Stack Exchange
I think if I'm reading the loop item that's what I'm looking for. Though I'm not sure why ansible is refusing to perform the loop. Any help about how to merge lists of objects together with optional attributes that could or could not be set because k8s is pretty picky about what's being set ... More on devops.stackexchange.com
🌐 devops.stackexchange.com
Ansible: Merge 2 lists using common key - Stack Overflow
I have a use case where I need to merge 2 lists on common key name using Ansible. List1: { "poc-cu2": [ "40:A6:B7:5E:22:11", "40:A6:B7:5E:2... More on stackoverflow.com
🌐 stackoverflow.com
Ansible: Combining 2 Lists by a Specific Attribute - Stack Overflow
I have 2 lists that I'm trying to combine together, by a specific attribute (the "device" attribute). I am not instantiating the lists in my code anywhere, they are being populated by API... More on stackoverflow.com
🌐 stackoverflow.com
jinja2 - Append list variable to another list in Ansible - Stack Overflow
is it possible to append a variable list to a static list in ansible? I can define the whole list as a variable: my_list: - 1 - 2 - 3 and then use it in a playbook as something: {{my_list... More on stackoverflow.com
🌐 stackoverflow.com
🌐
Reddit
reddit.com › r/ansible › combine and merge lists... how?
r/ansible on Reddit: Combine and Merge Lists... How?
February 2, 2021 -

Hi all,

Looking for some assistance please...

I am attempting to not only append two lists together ( varCombined: "{{ var1 + var2}}" ) but also merge the lists so that there are no duplicates.

My playbook is as follows:

- name: testing
  hosts: localhost
  tasks:
    - name: set vars
      set_fact:
        paths_blah:
          - includeFilePath: "/path1"
            excludeFilePaths:
              - "/path1/aaa"
              - "/path1/bbb"
          - includeFilePath: "/path2"
            excludeFilePaths:
              - "/path2/aaa"
        paths_blob:
          - includeFilePath: "/path3"
            excludeFilePaths:
              - "/path3/aaa"
              - "/path3/bbb"
          - includeFilePath: "/path1"
            excludeFilePaths:
              - "/path1/zzz"
        paths_berry:
          - includeFilePath: "/path4"
            excludeFilePaths:
              - "/path4/aaa"
              - "/path4/bbb"
          - includeFilePath: "/path2"
            excludeFilePaths:
              - "/path1/zzz"

    - set_fact:
        paths_combined: "{{ paths_blah + paths_blob + paths_berry }}"

    - name: print vars paths_combined
      debug:
        var: paths_combined

.. when run, the output of which is:

PLAY [print a message to screen] *****************************************************************************************************************

<snip>

TASK [print vars paths_combined] *****************************************************************************************************************
ok: [localhost] => {
    "paths_combined": [
        {
            "excludeFilePaths": [
                "/path1/aaa", 
                "/path1/bbb"
            ], 
            "includeFilePath": "/path1"
        }, 
        {
            "excludeFilePaths": [
                "/path2/aaa"
            ], 
            "includeFilePath": "/path2"
        }, 
        {
            "excludeFilePaths": [
                "/path3/aaa", 
                "/path3/bbb"
            ], 
            "includeFilePath": "/path3"
        }, 
        {
            "excludeFilePaths": [
                "/path1/zzz"
            ], 
            "includeFilePath": "/path1"
        }, 
        {
            "excludeFilePaths": [
                "/path4/aaa", 
                "/path4/bbb"
            ], 
            "includeFilePath": "/path4"
        }, 
        {
            "excludeFilePaths": [
                "/path1/zzz"
            ], 
            "includeFilePath": "/path2"
        }
    ]
}

.. you can see the lists are only appended, not merged: path1 and path2 appear twice in the list.

The desired output is a list with only unique items, merged rather than appended. So taking the above example the desired output would be:

paths_combined:
  - includeFilePath: "/path1"
    excludeFilePaths:
      - "/path1/aaa"
      - "/path1/bbb"
      - "/path1/zzz"
  - includeFilePath: "/path2"
    excludeFilePaths:
      - "/path2/aaa"
      - "/path2/zzz"
  - includeFilePath: "/path3"
    excludeFilePaths:
      - "/path3/aaa"
      - "/path3/bbb"
  - includeFilePath: "/path4"
    excludeFilePaths:
      - "/path4/aaa"
      - "/path4/bbb"

Can anybody suggest how I may accomplish this?

Thanks!

Top answer
1 of 2
1

loop requires a valid list. But we got here is,

[({'i_am': 'sam', 'eggs_ham': '456'}, AnsibleUndefined),
 ({'i_am': 'sam', 'eggs_ham': 'ham'}, {'transport': 'train'}),
 ({'i_am': 'eggs', 'eggs_ham': '456'}, AnsibleUndefined)]

Since no value is assigned to transport_fact_results when common_thing_2 is not defined.

if you remove the when condition it will work.

CODE:

- hosts: localhost
  vars:
    default_thing_1: 123
    default_thing_2: 456
    default_transport: train
    list_of_things:
      - name: foo
        common_thing_1: "sam"
      - name: bar
        common_thing_1: "sam"
        common_thing_2: "ham"
      - name: biz
        common_thing_1: "eggs"
  tasks:
    - name: Set the Things Facts
      set_fact:
        thing:
          i_am: "{{ item.common_thing_1 | default(default_thing_1) }}"
          eggs_ham: "{{ item.common_thing_2 | default(default_thing_2) }}"
      loop: "{{ list_of_things }}"
      register: things_fact_results
    - name: Set Optional Things Facts
      set_fact:
        thing:
          transport: "{{ item.transport | default(default_transport) }}"
      when: "item.common_thing_2 is defined and item.common_thing_2 == 'ham'"
      loop: "{{ list_of_things }}"
      register: transport_fact_results
    - name: Debug the Facts
      debug:
        msg: "{{ item.0 | combine(item.1|default({})) }}"
      loop: "{{
        things_fact_results.results |
        map(attribute='ansible_facts.thing') | list |
        zip(
          transport_fact_results.results |
          map(attribute='ansible_facts.thing') | list
        ) | list
      }}"

OUTPUT:

PLAY [localhost] ************************************************************************************************************************************************************************************************

TASK [Gathering Facts] ******************************************************************************************************************************************************************************************
ok: [localhost]

TASK [Set the Things Facts] *************************************************************************************************************************************************************************************
ok: [localhost] => (item={u'common_thing_1': u'sam', u'name': u'foo'})
ok: [localhost] => (item={u'common_thing_1': u'sam', u'common_thing_2': u'ham', u'name': u'bar'})
ok: [localhost] => (item={u'common_thing_1': u'eggs', u'name': u'biz'})

TASK [Set Optional Things Facts] ********************************************************************************************************************************************************************************
ok: [localhost] => (item={u'common_thing_1': u'sam', u'name': u'foo'})
ok: [localhost] => (item={u'common_thing_1': u'sam', u'common_thing_2': u'ham', u'name': u'bar'})
ok: [localhost] => (item={u'common_thing_1': u'eggs', u'name': u'biz'})

TASK [Debug the Facts] ******************************************************************************************************************************************************************************************
ok: [localhost] => (item=[{u'eggs_ham': u'456', u'i_am': u'sam'}, {u'transport': u'train'}]) => {
    "msg": {
        "eggs_ham": "456", 
        "i_am": "sam", 
        "transport": "train"
    }
}
ok: [localhost] => (item=[{u'eggs_ham': u'ham', u'i_am': u'sam'}, {u'transport': u'train'}]) => {
    "msg": {
        "eggs_ham": "ham", 
        "i_am": "sam", 
        "transport": "train"
    }
}
ok: [localhost] => (item=[{u'eggs_ham': u'456', u'i_am': u'eggs'}, {u'transport': u'train'}]) => {
    "msg": {
        "eggs_ham": "456", 
        "i_am": "eggs", 
        "transport": "train"
    }
}

PLAY RECAP ******************************************************************************************************************************************************************************************************
localhost                  : ok=4    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0  
2 of 2
0

So the way I figured it out was to put the when condition into Jinja.

- hosts: localhost
  vars:
    default_thing_1: 123
    default_thing_2: 456
    default_transport: train
    list_of_things:
      - name: foo
        common_thing_1: "sam"
      - name: bar
        common_thing_1: "sam"
        common_thing_2: "ham"
      - name: biz
        common_thing_1: "eggs"
  tasks:
    - name: Set the Things Facts
      set_fact:
        thing:
          i_am: "{{ item.common_thing_1 | default(default_thing_1) }}"
          eggs_ham: "{{ item.common_thing_2 | default(default_thing_2) }}"
      loop: "{{ list_of_things }}"
      register: things_fact_results
    - name: Set Optional Things Facts
      set_fact:
        thing: |
          {% if item.common_thing_2 is defined and item.common_thing_2 == 'ham' %}
          { "transport": "{{ item.transport | default(default_transport) }}" }
          {% else %}
          {}
          {% endif %}
      loop: "{{ list_of_things }}"
      register: transport_fact_results
    - name: Debug the Facts
      debug:
        msg: "{{ item.0 | combine(item.1) }}"
      loop: "{{
        things_fact_results.results |
        map(attribute='ansible_facts.thing') | list |
        zip(
          transport_fact_results.results |
          map(attribute='ansible_facts.thing') | list
        ) | list
      }}"

This is kinda sloppy but does allow me to specify the else condition to set the result over every iteration to something instead of AnsibleUndefined.

I don't really like this solution but it works.

$ ansible-playbook playbook.yml 
[WARNING]: No inventory was parsed, only implicit localhost is available
[WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all'

PLAY [localhost] *******************************************************************************************************************************************************************

TASK [Gathering Facts] *************************************************************************************************************************************************************
ok: [localhost]

TASK [Set the Things Facts] ********************************************************************************************************************************************************
ok: [localhost] => (item={'name': 'foo', 'common_thing_1': 'sam'})
ok: [localhost] => (item={'name': 'bar', 'common_thing_1': 'sam', 'common_thing_2': 'ham'})
ok: [localhost] => (item={'name': 'biz', 'common_thing_1': 'eggs'})

TASK [Set Optional Things Facts] ***************************************************************************************************************************************************
ok: [localhost] => (item={'name': 'foo', 'common_thing_1': 'sam'})
ok: [localhost] => (item={'name': 'bar', 'common_thing_1': 'sam', 'common_thing_2': 'ham'})
ok: [localhost] => (item={'name': 'biz', 'common_thing_1': 'eggs'})

TASK [Debug the Facts] *************************************************************************************************************************************************************
ok: [localhost] => (item=[{'i_am': 'sam', 'eggs_ham': '456'}, {}]) => {
    "msg": {
        "eggs_ham": "456",
        "i_am": "sam"
    }
}
ok: [localhost] => (item=[{'i_am': 'sam', 'eggs_ham': 'ham'}, {'transport': 'train'}]) => {
    "msg": {
        "eggs_ham": "ham",
        "i_am": "sam",
        "transport": "train"
    }
}
ok: [localhost] => (item=[{'i_am': 'eggs', 'eggs_ham': '456'}, {}]) => {
    "msg": {
        "eggs_ham": "456",
        "i_am": "eggs"
    }
}

PLAY RECAP *************************************************************************************************************************************************************************
localhost                  : ok=4    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   

Now the transport key is defined in the one object but not the other. So I can pass the item off to a complex object that's part of an API call.

🌐
Ansible
docs.ansible.com › ansible › latest › collections › ansible › builtin › union_filter.html
ansible.builtin.union filter – union of lists — Ansible Community Documentation
Items in the resulting list are returned in arbitrary order. This describes the input of the filter, the value before | ansible.builtin.union. This describes keyword parameters of the filter. These are the values key1=value1, key2=value2 and so on in the following example: input | ansible.builtin.union(key1=value1, key2=value2, ...) ... # return the unique elements of list1 added to list2 # list1: [1, 2, 5, 1, 3, 4, 10] # list2: [1, 2, 3, 4, 5, 11, 99] {{ list1 | union(list2) }} # => [1, 2, 5, 3, 4, 10, 11, 99]
🌐
FreeKB
freekb.net › Article
Ansible - Combine or Merge a List using plus or zip
The debug module can be used to output the entire list, like this. --- - hosts: localhost tasks: - set_fact: fruit: [ apple, banana, orange, grapes ] veggy: [ onion, pepper, tomato, carrot ] - debug: var: fruit - debug: var: veggy ... Which should return the following. ok: [localhost] => { "fruit": [ "apple" "banana", "orange", "grapes" ] } ok: [localhost] => { "veggy": [ "onion" "pepper", "tomato", "carrot" ] } Here is how you could combine or merge these lists into a new list.
🌐
Ansible
docs.ansible.com › ansible › latest › collections › ansible › builtin › together_lookup.html
ansible.builtin.together lookup – merges lists into synchronized list — Ansible Community Documentation
- name: item.0 returns from the 'a' list, item.1 returns from the '1' list ansible.builtin.debug: msg: "{{ item.0 }} and {{ item.1 }}" with_together: - ['a', 'b', 'c', 'd'] - [1, 2, 3, 4]
🌐
Packetswitch
packetswitch.co.uk › ansible-lists-examples
Working with List in Ansible
March 4, 2024 - The final task uses the debug module ... items that are different between two lists in Ansible, you can use a combination of Jinja2 filters: difference and unique....
Find elsewhere
🌐
Ansible
docs.ansible.com › ansible › latest › collections › ansible › builtin › unique_filter.html
ansible.builtin.unique filter – set of unique items of a list — Ansible Community Documentation
These are the values key1=value1, key2=value2 and so on in the following example: input | ansible.builtin.unique(key1=value1, key2=value2, ...) ... # return only the unique elements of list1 # list1: [1, 2, 5, 1, 3, 4, 10] {{ list1 | unique }} # => [1, 2, 5, 3, 4, 10] # return case sensitive unique elements {{ ['a', 'A', 'a'] | unique(case_sensitive='true') }} # => ['a', 'A'] # return case insensitive unique elements {{ ['b', 'B', 'b'] | unique() }} # => ['b'] # return unique elements of list based on attribute # => [{"age": 12, "name": "a" }, { "age": 14, "name": "b"}] - debug: msg: "{{ sample | unique(attribute='age') }}" vars: sample: - name: a age: 12 - name: b age: 14 - name: c age: 14
🌐
GitHub
github.com › leapfrogonline › ansible-merge-vars
GitHub - leapfrogonline/ansible-merge-vars: An Ansible action plugin to explicitly merge inventory variables · GitHub
This plugin has an option to dedup lists when creating the merged var. You can unique filter with the new lookup for that. This plugin has the option to non-recursively merge dicts.
Starred by 105 users
Forked by 18 users
Languages   Python 96.9% | Makefile 3.1%
🌐
Ansible
docs.ansible.com › projects › ansible › 5 › collections › community › general › docsite › filter_guide_abstract_informations_merging_lists_of_dictionaries.html
Merging lists of dictionaries — Ansible Documentation
November 7, 2022 - Quoting: “Ansible output that can be quite a bit easier to read than the default JSON formatting.” See the documentation for the community.general.yaml callback plugin. Let us use the lists below in the following examples: list1: - name: foo extra: true - name: bar extra: false - name: meh extra: true list2: - name: foo path: /foo - name: baz path: /baz · In the example below the lists are merged by the attribute name:
🌐
AnsiblePilot
ansiblepilot.com › home › articles › ansible product vs combine: list cartesian product & dict merging
Ansible Combine Filter: Merge Dictionaries & Data Structures
January 1, 2024 - `yaml Generate all test combinations • vars: python_versions: ["3.10", "3.11", "3.12"] ansible_versions: ["2.16", "2.17"] debug: msg: "Test Python {{ item.0 }} + Ansible {{ item.1 }}" loop: "{{ python_versions | product(ansible_versions) | list }}" ` ... `yaml • vars: a: { x: 1 } b: { y: 2 } c: { z: 3 } debug: msg: "{{ a | combine(b, c) }}" { x: 1, y: 2, z: 3 } ` ... By default, it overwrites nested dicts entirely. Use recursive=true to deep-merge nested dictionaries.
Top answer
1 of 2
2

You can get most of what you want using the combine filter, like this:

- hosts: localhost
  gather_facts: false
  tasks:
    - set_fact:
        dict3: "{{ dict1|combine(dict2, list_merge='append') }}"

    - debug:
        var: dict3

This will produce:

TASK [debug] *******************************************************************
ok: [localhost] => {
    "dict3": {
        "poc-cu2": [
            "40:A6:B7:5E:22:11",
            "40:A6:B7:5E:22:22",
            "root",
            "9WKA3KK3XN39",
            "9.3.13.44"
        ],
        "test2211": [
            "40:A6:B7:5E:33:11",
            "40:A6:B7:5E:33:22",
            "root2211",
            "221122112211",
            "9.3.13.82"
        ],
        "test2244": [
            "40:A6:B7:5E:22:45",
            "40:A6:B7:5E:22:46"
        ]
    }
}

If you want the final result to consist of only the keys common to both dictionaries it gets a little trickier, but this seems to work:

- hosts: localhost
  gather_facts: false
  tasks:
    - set_fact:
        dict3: "{{ dict3|combine({item: dict1[item] + dict2[item]}) }}"
      when: item in dict2
      loop: "{{ dict1.keys() }}"
      vars:
        dict3: {}

    - debug:
        var: dict3

Which produces:

TASK [debug] *******************************************************************
ok: [localhost] => {
    "dict3": {
        "poc-cu2": [
            "40:A6:B7:5E:22:11",
            "40:A6:B7:5E:22:22",
            "root",
            "9WKA3KK3XN39",
            "9.3.13.44"
        ],
        "test2211": [
            "40:A6:B7:5E:33:11",
            "40:A6:B7:5E:33:22",
            "root2211",
            "221122112211",
            "9.3.13.82"
        ]
    }
}

The above works by iterating over the keys in dict1, and for each key from dict1 that also exists in dict2, we synthesize a new dictionary containing the corresponding values from both dict1 and dict2 and then merge it into our final dictionary using the combine filter.

2 of 2
0

Given the data

    dict1:
      poc-cu2:
        - 40:A6:B7:5E:22:11
        - 40:A6:B7:5E:22:22
      test2211:
        - 40:A6:B7:5E:33:11
        - 40:A6:B7:5E:33:22
      test2244:
        - 40:A6:B7:5E:22:45
        - 40:A6:B7:5E:22:46

    dict2:
      poc-cu2:
        - root
        - 9WKA3KK3XN39
        - 9.3.13.44
      test2211:
        - root2211
        - '221122112211'
        - 9.3.13.82

Iteration is not needed. Put the declarations below as appropriate. The dictionary dict_cmn keeps the merged common attributes of dict1 and dict2

dict_1_2: "{{ dict1|combine(dict2, list_merge='append') }}"
keys_cmn: "{{ dict1.keys()|intersect(dict2.keys()) }}"
vals_cmn: "{{ keys_cmn|map('extract', dict_1_2) }}"
dict_cmn: "{{ dict(keys_cmn|zip(vals_cmn)) }}"

give

  dict_1_2:
    poc-cu2:
    - 40:A6:B7:5E:22:11
    - 40:A6:B7:5E:22:22
    - root
    - 9WKA3KK3XN39
    - 9.3.13.44
    test2211:
    - 40:A6:B7:5E:33:11
    - 40:A6:B7:5E:33:22
    - root2211
    - '221122112211'
    - 9.3.13.82
    test2244:
    - 40:A6:B7:5E:22:45
    - 40:A6:B7:5E:22:46

  keys_cmn:
  - poc-cu2
  - test2211

  vals_cmn:
  - - 40:A6:B7:5E:22:11
    - 40:A6:B7:5E:22:22
    - root
    - 9WKA3KK3XN39
    - 9.3.13.44
  - - 40:A6:B7:5E:33:11
    - 40:A6:B7:5E:33:22
    - root2211
    - '221122112211'
    - 9.3.13.82

  dict_cmn:
    poc-cu2:
    - 40:A6:B7:5E:22:11
    - 40:A6:B7:5E:22:22
    - root
    - 9WKA3KK3XN39
    - 9.3.13.44
    test2211:
    - 40:A6:B7:5E:33:11
    - 40:A6:B7:5E:33:22
    - root2211
    - '221122112211'
    - 9.3.13.82
Top answer
1 of 2
1

There are more options.

  • Iterate the lists if list1 is properly sorted and provides exactly what you need
    - set_fact:
        result: "{{ result|d([]) + _item }}"
      with_together:
        - "{{ list1.0 }}"
        - "{{ list2.0|groupby('device')|map(attribute=1)|list }}"
      vars:
        _item: "{{ [item.0]|product(item[1:])|map('combine')|list }}"

gives the desired result

result:
  - device: LTM1_Device
    host: /Common/LTM1_Host1
    ip: 0.0.0.1
    link: LTM1_Link
    ltm_pool: LTM1_Pool
    port: '5555'
  - device: LTM1_Device
    host: /Common/LTM1_Host2
    ip: 0.0.0.2
    link: LTM1_Link
    ltm_pool: LTM1_Pool
    port: '5555'
  - device: LTM2_Device
    host: /Common/LTM2_Host1
    ip: 0.0.0.3
    link: LTM2_Link
    ltm_pool: LTM2_Pool
    port: '5555'
  - device: LTM2_Device
    host: /Common/LTM2_Host2
    ip: 0.0.0.4
    link: LTM2_Link
    ltm_pool: LTM2_Pool
    port: '5555'
  - device: LTM3_Device
    host: /Common/LTM3_Host1
    ip: 0.0.0.5
    link: LTM3_Link
    ltm_pool: LTM3_Pool
    port: '5555'
  - device: LTM3_Device
    host: /Common/LTM3_Host2
    ip: 0.0.0.6
    link: LTM3_Link
    ltm_pool: LTM3_Pool
    port: '5555'

(details)

  • The next option is converting list1 to dictionaries
links: "{{ list1.0|items2dict(key_name='device', value_name='link') }}"
pools: "{{ list1.0|items2dict(key_name='device', value_name='ltm_pool') }}"

give

links:
  LTM1_Device: LTM1_Link
  LTM2_Device: LTM2_Link
  LTM3_Device: LTM3_Link
pools:
  LTM1_Device: LTM1_Pool
  LTM2_Device: LTM2_Pool
  LTM3_Device: LTM3_Pool

Iterate list2 and combine the dictionaries. The task below gives also the desired result

    - set_fact:
        result: "{{ result|d([]) + [_item] }}"
      loop: "{{ list2.0 }}"
      vars:
        _item: "{{ item|combine({'link': links[item.device]})|
                        combine({'ltm_pool': pools[item.device]}) }}"

(details)


Notes

  1. Example of a complete playbook
- hosts: localhost
  vars:
    list1:
      - - device: LTM1_Device
          link: LTM1_Link
          ltm_pool: LTM1_Pool
        - device: LTM2_Device
          link: LTM2_Link
          ltm_pool: LTM2_Pool
        - device: LTM3_Device
          link: LTM3_Link
          ltm_pool: LTM3_Pool
    list2:
      - - device: LTM1_Device
          host: /Common/LTM1_Host1
          ip: 0.0.0.1
          port: '5555'
        - device: LTM1_Device
          host: /Common/LTM1_Host2
          ip: 0.0.0.2
          port: '5555'
        - device: LTM2_Device
          host: /Common/LTM2_Host1
          ip: 0.0.0.3
          port: '5555'
        - device: LTM2_Device
          host: /Common/LTM2_Host2
          ip: 0.0.0.4
          port: '5555'
        - device: LTM3_Device
          host: /Common/LTM3_Host1
          ip: 0.0.0.5
          port: '5555'
        - device: LTM3_Device
          host: /Common/LTM3_Host2
          ip: 0.0.0.6
          port: '5555'
    links: "{{ list1.0|items2dict(key_name='device', value_name='link') }}"
    pools: "{{ list1.0|items2dict(key_name='device', value_name='ltm_pool') }}"
  tasks:
    - set_fact:
        result: "{{ result|d([]) + [_item] }}"
      loop: "{{ list2.0 }}"
      vars:
        _item: "{{ item|combine({'link': links[item.device]})|
                        combine({'ltm_pool': pools[item.device]}) }}"
    - debug:
        var: result

  1. If you want to avoid iteration create lists of the attributes ip and device
ip: "{{ list2.0|map(attribute='ip')|list }}"
device: "{{ list2.0|map(attribute='device')|list }}"

give

ip: [0.0.0.1, 0.0.0.2, 0.0.0.3, 0.0.0.4, 0.0.0.5, 0.0.0.6]
device: [LTM1_Device, LTM1_Device, LTM2_Device, LTM2_Device, LTM3_Device, LTM3_Device]

Then use the list device and extract both links and pools

link: "{{ device|map('extract', links)|list }}"
pool: "{{ device|map('extract', pools)|list }}"

give

link: [LTM1_Link, LTM1_Link, LTM2_Link, LTM2_Link, LTM3_Link, LTM3_Link]
pool: [LTM1_Pool, LTM1_Pool, LTM2_Pool, LTM2_Pool, LTM3_Pool, LTM3_Pool]

Create dictionaries

link_dict: "{{ dict(ip|zip(link)) }}"
pool_dict: "{{ dict(ip|zip(pool)) }}"

give

link_dict:
  0.0.0.1: LTM1_Link
  0.0.0.2: LTM1_Link
  0.0.0.3: LTM2_Link
  0.0.0.4: LTM2_Link
  0.0.0.5: LTM3_Link
  0.0.0.6: LTM3_Link
pool_dict:
  0.0.0.1: LTM1_Pool
  0.0.0.2: LTM1_Pool
  0.0.0.3: LTM2_Pool
  0.0.0.4: LTM2_Pool
  0.0.0.5: LTM3_Pool
  0.0.0.6: LTM3_Pool

and convert the dictionaries to lists

link_list: "{{ link_dict|dict2items(key_name='ip', value_name='link') }}"
pool_list: "{{ pool_dict|dict2items(key_name='ip', value_name='ltm_pool') }}"

give

link_list:
  - {ip: 0.0.0.1, link: LTM1_Link}
  - {ip: 0.0.0.2, link: LTM1_Link}
  - {ip: 0.0.0.3, link: LTM2_Link}
  - {ip: 0.0.0.4, link: LTM2_Link}
  - {ip: 0.0.0.5, link: LTM3_Link}
  - {ip: 0.0.0.6, link: LTM3_Link}
pool_list:
  - {ip: 0.0.0.1, ltm_pool: LTM1_Pool}
  - {ip: 0.0.0.2, ltm_pool: LTM1_Pool}
  - {ip: 0.0.0.3, ltm_pool: LTM2_Pool}
  - {ip: 0.0.0.4, ltm_pool: LTM2_Pool}
  - {ip: 0.0.0.5, ltm_pool: LTM3_Pool}
  - {ip: 0.0.0.6, ltm_pool: LTM3_Pool}

Finally, use filter community.general.lists_mergeby and merge the lists by attribute ip. This gives the desired result

result: "{{ [list2.0, link_list, pool_list]|community.general.lists_mergeby('ip') }}"

Example of a complete playbook

- hosts: localhost
  vars:
    list1:
      - - device: LTM1_Device
          link: LTM1_Link
          ltm_pool: LTM1_Pool
        - device: LTM2_Device
          link: LTM2_Link
          ltm_pool: LTM2_Pool
        - device: LTM3_Device
          link: LTM3_Link
          ltm_pool: LTM3_Pool
    list2:
      - - device: LTM1_Device
          host: /Common/LTM1_Host1
          ip: 0.0.0.1
          port: '5555'
        - device: LTM1_Device
          host: /Common/LTM1_Host2
          ip: 0.0.0.2
          port: '5555'
        - device: LTM2_Device
          host: /Common/LTM2_Host1
          ip: 0.0.0.3
          port: '5555'
        - device: LTM2_Device
          host: /Common/LTM2_Host2
          ip: 0.0.0.4
          port: '5555'
        - device: LTM3_Device
          host: /Common/LTM3_Host1
          ip: 0.0.0.5
          port: '5555'
        - device: LTM3_Device
          host: /Common/LTM3_Host2
          ip: 0.0.0.6
          port: '5555'
    links: "{{ list1.0|items2dict(key_name='device', value_name='link') }}"
    pools: "{{ list1.0|items2dict(key_name='device', value_name='ltm_pool') }}"
    ip: "{{ list2.0|map(attribute='ip')|list }}"
    device: "{{ list2.0|map(attribute='device')|list }}"
    link: "{{ device|map('extract', links)|list }}"
    pool: "{{ device|map('extract', pools)|list }}"
    link_dict: "{{ dict(ip|zip(link)) }}"
    pool_dict: "{{ dict(ip|zip(pool)) }}"
    link_list: "{{ link_dict|dict2items(key_name='ip', value_name='link') }}"
    pool_list: "{{ pool_dict|dict2items(key_name='ip', value_name='ltm_pool') }}"
    result: "{{ [list2.0, link_list, pool_list]|community.general.lists_mergeby('ip') }}"
  tasks:
    - debug:
        var: result

(details)


  1. If you can't or don't want to use community.general.lists_mergeby zip the lists and combine items. This gives also the desired result
result: "{{ list2.0|zip(link_list)|zip(pool_list)|map('flatten')|map('combine')|list }}"
2 of 2
0

With the help of a peer, I was able to figure it out by looping the second list, adding a selectattr filter to match my attribute against the first list, and combining that result with the second list to create a new list of results.

Filtered Loop:

- name: Combine lists
  set_fact:
    new_list: "{{ new_list | default([]) + [item|combine(filter_time)] }}"
  loop: "{{ list2 }}"
  vars:
    filter_time: "{{ list1 | selectattr('device', '==', item.device) | first }}"

Output:

ok: [localhost] => {
    "msg": [
        [
            {
                "device": "LTM1_Device",
                "host": "/Common/LTM1_Host1",
                "ip": "0.0.0.1",
                "link": "LTM1_Link",
                "ltm_pool": "LTM1_Pool",
                "port": "5555"
            },
            {
                "device": "LTM1_Device",
                "host": "/Common/LTM1_Host2",
                "ip": "0.0.0.2",
                "link": "LTM1_Link",
                "ltm_pool": "LTM1_Pool",
                "port": "5555"
            },
            {
                "device": "LTM2_Device",
                "host": "/Common/LTM2_Host1",
                "ip": "0.0.0.3",
                "link": "LTM2_Link",
                "ltm_pool": "LTM2_Pool",
                "port": "5555"
            },
            {
                "device": "LTM2_Device",
                "host": "/Common/LTM2_Host2",
                "ip": "0.0.0.4",
                "link": "LTM2_Link",
                "ltm_pool": "LTM2_Pool",
                "port": "5555"
            },
            {
                "device": "LTM3_Device",
                "host": "/Common/LTM3_Host1",
                "ip": "0.0.0.5",
                "link": "LTM3_Link",
                "ltm_pool": "LTM3_Pool",
                "port": "5555"
            },
            {
                "device": "LTM3_Device",
                "host": "/Common/LTM3_Host2",
                "ip": "0.0.0.6",
                "link": "LTM3_Link",
                "ltm_pool": "LTM3_Pool",
                "port": "5555"
            }
        ]
    ]
}
Top answer
1 of 2
1

I asked about lists vs dictionaries in the comment because of the impact it will have on the solution. If you were to restructure your data like this:

dict1:
  alice: ['role1', 'role2']
  bob: ['role1']

dict2:
  alice: ['role3']
  charlie: ['role2']

Then your solution becomes:

- set_fact:
    dict3: >-
      {{
      dict3|default([])|combine({
      item: (dict1[item]|default([]) + dict2[item]|default([]))|unique
      })
      }}
  loop: "{{ (dict1.keys()|list + dict2.keys()|list)|unique }}"

- debug:
    var: dict3

Which outputs:

TASK [debug] **********************************************************************************************************************************************************************************
ok: [localhost] => {
    "dict3": {
        "alice": [
            "role1", 
            "role2", 
            "role3"
        ], 
        "bob": [
            "role1"
        ], 
        "charlie": [
            "role2"
        ]
    }
}

If you're stuck with using lists, we can improve upon the json_query solution that Zeitounator suggested:

- set_fact:
    list3: >-
      {{
      list3|default([]) + [{
      'name': item,
      'roles': (list1|json_query('[?name==`' + item + '`].roles[]') + list2|json_query('[?name==`' + item + '`].roles[]'))|unique
      }]
      }}
  loop: "{{ (list1|json_query('[].name') + list2|json_query('[].name'))|unique }}"

- debug:
    var: list3

This produces your desired output:

TASK [debug] **********************************************************************************************************************************************************************************
ok: [localhost] => {
    "list3": [
        {
            "name": "alice", 
            "roles": [
                "role1", 
                "role2", 
                "role3"
            ]
        }, 
        {
            "name": "bob", 
            "roles": [
                "role1"
            ]
        }, 
        {
            "name": "charlie", 
            "roles": [
                "role2"
            ]
        }
    ]
}
2 of 2
0

This is a solution in plain ansible. If it becomes out of control because of your datastructure growth, you should consider writing your own filter (example)

---
- name: demo playbook for deeps dictionary merge
  hosts: localhost
  gather_facts: false

  vars:
    list1:
      - name: alice
        roles: ['role1', 'role2']
      - name: bob
        roles: ['role1']

    list2:
      - name: alice
        roles: ['role3']
      - name: charlie
        roles: ['role2']

  tasks:

    - debug:
        msg: >-
          roles for {{ item }}:
          {{
            (list1 | json_query("[?name == '" + item +"'].roles")).0 | default([])
            +
            (list2 | json_query("[?name == '" + item +"'].roles")).0 | default([])
          }}
      loop: >-
        {{
          (
            list1 | json_query("[].name")
            +
            list2 | json_query("[].name")
          )
          | unique
        }}

Which gives:

PLAY [demo playbook for deeps dictionary merge] ************************************************************************************************************************************************************************************************************************

TASK [debug] ************************************************************************************************************************************************************************************************************************************************************
Wednesday 24 April 2019  16:23:07 +0200 (0:00:00.046)       0:00:00.046 ******* 
ok: [localhost] => (item=alice) => {
    "msg": "roles for alice: ['role1', 'role2', 'role3']"
}
ok: [localhost] => (item=bob) => {
    "msg": "roles for bob: ['role1']"
}
ok: [localhost] => (item=charlie) => {
    "msg": "roles for charlie: ['role2']"
}
🌐
Blue-bag
blue-bag.com › blog › ansible-filters-taming-lists-part-1
Ansible filters for taming lists (part 1) | Blue-Bag
March 27, 2015 - This says take the dictionary structure (images_to_update) and create a list from the 'type' attribute and further make it unique - Cool huh!
🌐
Ansible
docs.ansible.com › projects › ansible › latest › collections › community › general › docsite › filter_guide_abstract_informations_merging_lists_of_dictionaries.html
Merging lists of dictionaries — Ansible Community Documentation
If you have two or more lists of dictionaries and want to combine them into a list of merged dictionaries, where the dictionaries are merged by an attribute, you can use the community.general.lists_mergeby filter. ... The output of the examples in this section use the YAML callback plugin. Quoting: “Ansible output that can be quite a bit easier to read than the default JSON formatting.” See the documentation for the community.general.yaml callback plugin.