Duplicating an SPN

[zkytony]

TL;DR: how should I duplicate an SPN? I have a big IVs and I want the duplicated SPNs to take different subsets of indicators from this big IVs. How should I do that?

I am now on commit a649c62 (09/10/2018) "batch noise now shuffles instead of rolls".

In a prior version of the libspn that I used (on the master branch), I was able to duplicate an SPN by using my own compute_graph_up function, pasted below. The key difference is that this function deals with the case where there are indicator variables specified in the format of a tuple (node, indices).

from collections import deque, defaultdict
def mod_compute_graph_up(root, val_fun, **kwargs):
    all_values = {}
    stack = deque()  # Stack of inputs to process                                                                                                                                                                                                                               
    stack.append((root, None))  # node and index                                                                                                                                                                                                                                

    while stack:
        next_input = stack[-1]
        # Was this node already processed?                                                                                                                                                                                                                                      
        # This might happen if the node is referenced by several parents                                                                                                                                                                                                        
        if next_input not in all_values:
            if next_input[0].is_op:
                # OpNode                                                                                                                                                                                                                                                        
                input_vals = []  # inputs to the node of 'next_input'                                                                                                                                                                                                           
                all_input_vals = True
                # Gather input values for non-const val fun                                                                                                                                                                                                                     
                for inpt in next_input[0].inputs:
                    if inpt:  # Input is not empty                                                                                                                                                                                                                              
                        try:
                            # Check if input_node in all_vals                                                                                                                                                                                                                   
                            if inpt.indices is None:
                                input_vals.append(all_values[(inpt.node, None)])
                            else:
                                input_vals.append(all_values[(inpt.node, tuple(inpt.indices))])
                        except KeyError:
                            all_input_vals = False
                            if inpt.indices is None:
                                stack.append((inpt.node, None))
                            else:
                                stack.append((inpt.node, tuple(inpt.indices)))
                    else:
                        # This input was empty, use None as value                                                                                                                                                                                                               
                        input_vals.append(None)
                # Got all inputs?                                                                                                                                                                                                                                               
                if all_input_vals:
                    last_val = val_fun(next_input, *input_vals, **kwargs)
                    all_values[next_input] = last_val
                    stack.pop()
            else:
                # VarNode, ParamNode                                                                                                                                                                                                                                            
                last_val = val_fun(next_input, **kwargs)
                all_values[next_input] = last_val
                stack.pop()
        else:
            stack.pop()

    return last_val

Then I wrote a function that uses mod_compute_graph_up which actually copies the entire structure and parameters of a given SPN, while keeping the same inputs. This worked well before, and I have documented how it works. But now after I have switched to newer code (a649c62), my SPN duplication code does not work any more. I got an error:

TypeError: unsupported operand type(s) for +: 'int' and 'NoneType'

I think this is due to some changes to the SPN structure internally. Then, how should I duplicate an SPN? I have a big IVs and I want the duplicated SPNs to take different subsets of indicators from this big IVs. How should I do that?

cc: @jostosh @pronobis

(apologies for keep posting issues)

[jostosh]

Could you show me the traceback of the error? Where in the code is it happening?

Also, I was wondering why you want to duplicate SPNs? Do they take in different IVs? In that case you could do:

1. Build first SPN and build TF graph
2. Replace only the bottom IVs and rebuild TF graph
3. Repeat 2 for as long as necessary

[zkytony]

I will post the traceback later, but for me right now, I would like to use a single IVs object and have different SPNs with the same structure and parameters take different subsets of indices from the IVs.

I am not sure how you to do step 2 as you described. Right now I construct the SPN structure via dense generation and calling dense generator again might result in a different structure. Therefore I need to copy the structure of a generated SPN.

[zkytony]

Another reason I was duplicating SPNs is because I need to infer the value of some indicators in the IVs given values of other indicators in this IVs. I thought it would be more straightforward to implement the inference if the duplicated SPNs form together a big SPN graph. This may not be the most optimal way to implement this but it is what my code is based on for now.

[zkytony]

Here is the traceback:

Partition 1
Will duplicate ThreeNodeTemplate 26 times.
Duplicating ThreeNodeTemplate... 1 Traceback (most recent call last):
  File "/home/zkytony/Documents/thesis/experiments/deep-semantic-mapping/deepsm/experiments/train_test_graphspn.py", line 33, in <module>
    main()
  File "/home/zkytony/Documents/thesis/experiments/deep-semantic-mapping/deepsm/experiments/train_test_graphspn.py", line 29, in main
    available_commands[args.command]()
  File "/home/zkytony/Documents/thesis/experiments/deep-semantic-mapping/deepsm/experiments/cold_database_experiment.py", line 660, in same_building
    seq_id=args.seq_id, skip_placeholders=args.skip_placeholders)
  File "/home/zkytony/Documents/thesis/experiments/deep-semantic-mapping/deepsm/experiments/cold_database_experiment.py", line 481, in run_experiment
    partitions=pset)
  File "/home/zkytony/Documents/thesis/experiments/deep-semantic-mapping/deepsm/graphspn/tbm/spn_instance.py", line 232, in __init__
    db_name=kwargs.get('db_name', None), extra_partition_multiplyer=kwargs.get('extra_partition_multiplyer', 1))
  File "/home/zkytony/Documents/thesis/experiments/deep-semantic-mapping/deepsm/graphspn/tbm/spn_instance.py", line 354, in _init_struct
    template_spn_roots.extend(NodeTemplateInstanceSpn._duplicate_template_spns(self, tspns, template, supergraph, nodes_covered))
  File "/home/zkytony/Documents/thesis/experiments/deep-semantic-mapping/deepsm/graphspn/tbm/spn_instance.py", line 426, in _duplicate_template_spns
    labels=[labels])
  File "/home/zkytony/Documents/thesis/experiments/deep-semantic-mapping/deepsm/graphspn/spn_model.py", line 395, in mod_compute_graph_up
    last_val = val_fun(next_input, *input_vals, **kwargs)
  File "/home/zkytony/Documents/thesis/experiments/deep-semantic-mapping/deepsm/graphspn/tbm/spn_template.py", line 278, in dup_fun_up
    return spn.Sum(*args[2:], weights=args[0])
  File "/home/zkytony/Documents/thesis/experiments/libspn/libspn/graph/sum.py", line 42, in __init__
    name=name)
  File "/home/zkytony/Documents/thesis/experiments/libspn/libspn/graph/basesum.py", line 63, in __init__
    self._reset_sum_sizes(num_sums=num_sums, sum_sizes=sum_sizes)
  File "/home/zkytony/Documents/thesis/experiments/libspn/libspn/graph/basesum.py", line 187, in _reset_sum_sizes
    self._sum_sizes = sum_sizes or self._get_sum_sizes(self._num_sums)
  File "/home/zkytony/Documents/thesis/experiments/libspn/libspn/graph/basesum.py", line 89, in _get_sum_sizes
    num_values = sum(input_sizes[2:])  # Skip ivs, weights
TypeError: unsupported operand type(s) for +: 'int' and 'NoneType'

[jostosh]

So, you need this feature to work on feature/convspn specifically? I will merge jostosh/libspn/dev-refactor-for-release to it soon which hopefully fixes this issue.

[zkytony]

I see. Sounds like you know why that happens?
I can also merge this branch to my local branch myself, if that creates less potential trouble for you.