Ansible, or jinja, has a few builtin filters to work with 2 lists, like intersect and difference and union, but they are all to work with simple lists. What I was looking for was the ability to merge dictionary lists and if it was somehow possible also do a join of properties.
I couldn't find it so I created one.
For whomever can't wait for the explanation, here is the code :
class FilterModule(object):
'''
force a list to a dictionary list
'''
def to_dict(self,obj,attr=""):
out = dict() # new dictionary
# we first force to dict
if(type(obj) is not dict):
temp = dict()
temp["name"] = obj
obj = temp
if(attr!=""): # if we need to force an attribute
out[attr]=obj # we assign a level deeper
else: # if no attribute name, we flatten it (could potentially overwrite attributes)
out.update(obj) # we update
return out
'''
joins 2 lists of dictionaries
'''
def join_dict_list(self,list1,list2,attr1="",attr2="",on="True"):
out = []
for a in list1:
for b in list2:
_item_ = dict()
_item_.update(self.to_dict(a,attr1))
_item_.update(self.to_dict(b,attr2))
if(eval(on.replace('`','"'))):
out.append(_item_)
return out
def filters(self):
return {
'to_dict':self.to_dict,
'join_dict_list':self.join_dict_list
}Copy this code as a python file (example : ansible_join_dict_lists.py) in a subfolder "filter_plugins". Read my article about how to create custom filter plugin.
Attributes :
- List1 : Your first dictionary list (mandatory ; and is pipe-input)
- List2 : Your second dictionary list (mandatory)
- Nest attribute1 : force object from list1 in a new object (optional)
- Nest attribute2 : force object from lits2 in a new object (optional)
- Join Python expression : a python expression to make an inner join. (optional)
Examples:
Example 1 : Flattened outer join
Playbook :
---
- name: This is a join example
hosts: localhost
vars:
list1:
- name: esxi1
ip: 10.0.0.1
- name: esxi2
ip: 10.0.0.2
list2:
- datastore: datastore1
path: /datastore1
- datastore: datastore2
path: /datastore2
tasks:
- name: "print join"
debug:
msg: "{{ list1 | join_dict_list(list2) }}"Output:
[
{
"datastore": "datastore1",
"ip": "10.0.0.1",
"name": "esxi1",
"path": "/datastore1"
},
{
"datastore": "datastore2",
"ip": "10.0.0.1",
"name": "esxi1",
"path": "/datastore2"
},
{
"datastore": "datastore1",
"ip": "10.0.0.2",
"name": "esxi2",
"path": "/datastore1"
},
{
"datastore": "datastore2",
"ip": "10.0.0.2",
"name": "esxi2",
"path": "/datastore2"
}
]
Example 2 : Nested outer join
Playbook :
---
- name: This is a join example
hosts: localhost
vars:
list1:
- name: esxi1
ip: 10.0.0.1
- name: esxi2
ip: 10.0.0.2
list2:
- datastore: datastore1
path: /datastore1
- datastore: datastore2
path: /datastore2
tasks:
- name: "print join"
debug:
msg: "{{ list1 | join_dict_list(list2,'esxhosts','datastores') }}"Output:
[
{
"datastores": {
"datastore": "datastore1",
"path": "/datastore1"
},
"esxhosts": {
"ip": "10.0.0.1",
"name": "esxi1"
}
},
{
"datastores": {
"datastore": "datastore2",
"path": "/datastore2"
},
"esxhosts": {
"ip": "10.0.0.1",
"name": "esxi1"
}
},
{
"datastores": {
"datastore": "datastore1",
"path": "/datastore1"
},
"esxhosts": {
"ip": "10.0.0.2",
"name": "esxi2"
}
},
{
"datastores": {
"datastore": "datastore2",
"path": "/datastore2"
},
"esxhosts": {
"ip": "10.0.0.2",
"name": "esxi2"
}
}
]
Example 3 : Semi nested outer join
Playbook :
---
- name: This is a join example
hosts: localhost
vars:
list1:
- name: esxi1
ip: 10.0.0.1
- name: esxi2
ip: 10.0.0.2
list2:
- name: datastore1
path: /datastore1
- name: datastore2
path: /datastore2
tasks:
- name: "print join"
debug:
msg: "{{ list1 | join_dict_list(list2,'','datastores') }}"Output:
[
{
"datastores": {
"name": "datastore1",
"path": "/datastore1"
},
"ip": "10.0.0.1",
"name": "esxi1"
},
{
"datastores": {
"name": "datastore2",
"path": "/datastore2"
},
"ip": "10.0.0.1",
"name": "esxi1"
},
{
"datastores": {
"name": "datastore1",
"path": "/datastore1"
},
"ip": "10.0.0.2",
"name": "esxi2"
},
{
"datastores": {
"name": "datastore2",
"path": "/datastore2"
},
"ip": "10.0.0.2",
"name": "esxi2"
}
]Example 4 : Flattened outer join with common attribue
Playbook :
---
- name: This is a join example
hosts: localhost
vars:
list1:
- name: esxi1
ip: 10.0.0.1
- name: esxi2
ip: 10.0.0.2
list2:
- name: datastore1
path: /datastore1
- name: datastore2
path: /datastore2
tasks:
- name: "print join"
debug:
msg: "{{ list1 | join_dict_list(list2) }}"Output:
[
{
"ip": "10.0.0.1",
"name": "datastore1",
"path": "/datastore1"
},
{
"ip": "10.0.0.1",
"name": "datastore2",
"path": "/datastore2"
},
{
"ip": "10.0.0.2",
"name": "datastore1",
"path": "/datastore1"
},
{
"ip": "10.0.0.2",
"name": "datastore2",
"path": "/datastore2"
}
]
As you can see, flattening with a common attribute will cause unwanted outcomes. As 'name' is a common attribute, the esxi name, in this case, gets dropped !
Example 5 : Nested inner join with join expression
PLaybook :
---
- name: This is a join example
hosts: localhost
vars:
list1:
- name: esxi1
ip: 10.0.0.1
- name: esxi2
ip: 10.0.0.2
list2:
- name: datastore1
path: /datastore1
esxi: esxi1
- name: datastore2
path: /datastore2
esxi: esxi2
tasks:
- name: "print join"
debug:
msg: "{{ list1 |
join_dict_list(
list2,
'esxihost',
'datastore',
'_item_[`esxihost`][`name`]==_item_[`datastore`][`esxi`]'
)
}}"Output:
[
{
"datastore": {
"esxi": "esxi1",
"name": "datastore1",
"path": "/datastore1"
},
"esxihost": {
"ip": "10.0.0.1",
"name": "esxi1"
}
},
{
"datastore": {
"esxi": "esxi2",
"name": "datastore2",
"path": "/datastore2"
},
"esxihost": {
"ip": "10.0.0.2",
"name": "esxi2"
}
}
]
PS 1 : As you can see in the playbook above, I use "_item_" to reference a merge object during the join. This is not really a special place holder, I actually use that variable in the Python code.
PS 2 : Since we use a Python-expression, you can infact manipulate the strings during the join expression, if that would be required.
PS 3 : Note the back-tic (`) character, which I use instead of escaping the quote character (\"). This is pure for better readability. In my Python code I replace the back-tic with a quote character again.
Example 6 : Join with regular list (non-dictionary list)
PLaybook :
---
- name: This is a join example
hosts: localhost
vars:
list1:
- name: esxi1
ip: 10.0.0.1
- name: esxi2
ip: 10.0.0.2
list2:
- datastore1
- datastore2
tasks:
- name: "print join"
debug:
msg: "{{ list1 | join_dict_list(list2,'esx','datastore') }}"Output:
[
{
"datastore": {
"name": "datastore1"
},
"esx": {
"ip": "10.0.0.1",
"name": "esxi1"
}
},
{
"datastore": {
"name": "datastore2"
},
"esx": {
"ip": "10.0.0.1",
"name": "esxi1"
}
},
{
"datastore": {
"name": "datastore1"
},
"esx": {
"ip": "10.0.0.2",
"name": "esxi2"
}
},
{
"datastore": {
"name": "datastore2"
},
"esx": {
"ip": "10.0.0.2",
"name": "esxi2"
}
}
]
As you can see, the regular list is converted to a dictionary with attribute "name".
0 Comments