Neural-symbolic Entity Matching

The example introduces the package’s entity matching functionality using neural-symbolic learning. The goals of the example are (i) to introduce the available options, (ii) discuss some aspects of the underlying model logic, and (iii) contrast the neural-symbolic functionality to that of purely deep learning entity matching. For an introduction to basic concepts and conventions of the package one may consult the Entity Matching with Similarity Maps and Deep Learning example (henceforth Example 1). For the reasoning capabilities of the package, set the Reasoning example.

Prerequisites

Load the libraries we will use and set the seed for reproducibility.

from neer_match.matching_model import NSMatchingModel
from neer_match.similarity_map import SimilarityMap
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
import random
import tensorflow as tf

random.seed(42)
np.random.seed(42)
tf.random.set_seed(42)

Preprocessing

We use the gaming data of Example 1. Similar to the deep learning case, the neural-symbolic functionality leverages similarity maps and expects the same structure of datasets as inputs. The concepts and requirements are detailed in Example 1, we abstain from detailing them here for brevity. In summary, our preprocessing stage constructs three datasets left, right, and matches, where left has 36 records, right has 39 records, and matches has the indices of the matching records in left and right.

left, right, matches = prepare_data()

Matching Model Setup

For simplicity, we employ the similarity map we used in Example 1.

instructions = {
    "title": ["jaro_winkler"],
    "platform": ["levenshtein", "discrete"],
    "year": ["euclidean", "discrete"],
    "developer~dev": ["jaro"]
}

similarity_map = SimilarityMap(instructions)
print(similarity_map)
SimilarityMap[
  ('title', 'title', 'jaro_winkler')
  ('platform', 'platform', 'levenshtein')
  ('platform', 'platform', 'discrete')
  ('year', 'year', 'euclidean')
  ('year', 'year', 'discrete')
  ('developer', 'dev', 'jaro')]

Matching models using neural-symbolic learning are constructed using the NSMatchingModel class. The constructor expects a similarity map object, exactly as in the pure deep learning case.

model = NSMatchingModel(similarity_map)

Although the exposed interface is similar to the deep learning model, the underlying class designs are different. The DLMatchingModel class inherits from the tensorflow.keras.Model class, while the NSMatchingModel uses a custom training loop. Nonetheless, the NSMatchingModel class exposes compile, fit, evaluate, predict, and suggest functions to ensure that the calling conventions in the user space are as consistent as possible between the two classes.

For instance, the compile method of the NSMatchingModel can be used to set the optimizer used during training.

model.compile(optimizer = tf.keras.optimizers.Adam(learning_rate = 0.001))

Model Training

The model is fitted using the fit function (see the fit documentation for details). The underline training loop of the neural-symbolic model is custom. Training treats the field pair and record pair models as fuzzy logic predicates. It adjusts their parameters to satisfy the following logic. First, for every matching example, all the field predicates and the record predicate should be (fuzzily) true. This rule is motivated by the observations that true record matches should in principle constitute matches in all subsets of their associated fields. Second, at least one of the field predicates and the record predicate should by (fuzzily) false for every non matching example. This rule is motivated by the observation that non matches should be distinct in at least one of their associated fields, because if they are not, then, in principle, they constitute a match.

The fit function expects left, right, and matches data sets. As in the deep learning case, the mismatch_share parameter controls the ratio of counterexamples to matches. The parameter satisfiability_weight is a float value between 0 and 1 that controls the weight of the satisfiability loss in the total loss function. The default value is 1.0, i.e., the training process considers only the fuzzy logic axioms. A value of 0.0 means that only the binary cross entropy loss is considered, which essentially reduces the model to a deep learning model. Any value between 0 and 1 trains a hybrid model that balances the two losses.

Finally, logging can be customized by setting the parameters verbose and log_mod_n. The verbose parameter controls whether the training process prints the loss values at each epoch. The log_mod_n parameter controls the frequency of the logging. For instance, if log_mod_n = 5, the training process logs the loss values every 5 epochs. The default value is 1.

model.fit(
    left,
    right,
    matches,
    epochs=100,
    batch_size=16,
    verbose=1,
    log_mod_n=10
)

Predictions and Suggestions

Matching predictions and suggestions follow the same calling conventions as corresponding deep learning member functions. Predictions can be obtained by calling predict and passing the left and right datasets. The returned prediction probabilities are stored in row-major order. First, the matching probabilities of the first row of left with all the rows of right are given. Then, the probabilities of the second row of left with all the rows of right are given, and so on. In total, the predict function returns a vector with rows equal to the product of the number of rows in the left and right data sets.

predictions = model.predict(left, right)

fig, ax = plt.subplots()
counts, bins= np.histogram(predictions, bins = 100)
cdf = np.cumsum(counts)/np.sum(counts)
ax.plot(bins[1:], cdf)
ax.set_xlabel("Matching Prediction")
ax.set_ylabel("Cumulative Density")
plt.show()

The suggest function also expects the left and right datasets as input. The function returns the top count matching predictions of the model for each row of the left dataset. The prediction probabilities of predict are grouped by the indices of the left dataset and sorted in descending order.

suggestions = model.suggest(left, right, count = 3)
suggestions["true_match"] = suggestions.loc[:, ["left", "right"]].apply(
    lambda x: any((x.left == matches.left) & (x.right==matches.right)), axis=1
)
suggestions = suggestions.join(
    matches.iloc[-no_duplicates:,:].assign(duplicate=True).set_index(['left', 'right']),
    on = ["left", "right"],
    how = "left"
)
suggestions.duplicate = suggestions.duplicate.apply(
    lambda x: False if pd.isna(x) else x
)
suggestions = suggestions.sort_values(
    by=["left", "prediction"], ascending=[True, False]
)

suggestions

left

right

prediction

true_match

duplicate

0

0

0

0.992840

True

False

1

0

14

0.992815

False

False

2

0

22

0.033318

False

False

3

1

1

0.992818

True

False

4

1

23

0.010145

False

False

5

1

10

0.010042

False

False

6

2

2

0.992842

True

False

7

2

36

0.025500

False

False

8

2

35

0.025500

False

False

9

3

3

0.992835

True

False

10

3

30

0.025500

False

False

11

3

21

0.010144

False

False

12

4

4

0.992842

True

False

13

4

12

0.992793

False

False

14

4

22

0.025500

False

False

15

5

5

0.992841

True

False

16

5

27

0.010758

False

False

17

5

15

0.010144

False

False

18

6

6

0.992839

True

False

19

6

24

0.010526

False

False

20

6

31

0.000821

False

False

21

7

7

0.992841

True

False

22

7

36

0.025500

False

False

23

7

35

0.025500

False

False

24

8

8

0.992838

True

False

25

8

9

0.011969

False

False

26

8

10

0.000838

False

False

27

9

9

0.992819

True

False

28

9

17

0.033318

False

False

29

9

28

0.025500

False

False

30

10

10

0.992824

True

False

31

10

23

0.992815

False

False

32

10

1

0.010009

False

False

33

11

11

0.992840

True

False

34

11

29

0.025500

False

False

35

11

14

0.010758

False

False

36

12

12

0.992826

True

False

37

12

4

0.992783

False

False

38

12

22

0.025500

False

False

39

13

13

0.992841

True

False

40

13

37

0.992832

True

True

41

13

33

0.992830

False

False

42

14

14

0.992816

True

False

43

14

0

0.992815

False

False

44

14

22

0.033318

False

False

45

15

15

0.992838

True

False

46

15

24

0.024185

False

False

47

15

27

0.010981

False

False

48

16

16

0.992815

True

False

49

16

17

0.010408

False

False

50

16

34

0.010386

False

False

51

17

17

0.992833

True

False

52

17

9

0.033318

False

False

53

17

21

0.025501

False

False

54

18

18

0.992842

True

False

55

18

36

0.992815

False

False

56

18

35

0.992815

False

False

57

19

19

0.992840

True

False

58

19

33

0.025504

False

False

59

19

13

0.025501

False

False

60

20

20

0.992842

True

False

61

20

32

0.992836

False

False

62

20

37

0.025500

False

False

63

21

21

0.992825

True

False

64

21

28

0.032934

False

False

65

21

17

0.025502

False

False

66

22

22

0.992836

True

False

67

22

14

0.033318

False

False

68

22

0

0.033318

False

False

69

23

23

0.992840

True

False

70

23

10

0.992769

False

False

71

23

1

0.010144

False

False

72

24

24

0.992815

True

False

73

24

15

0.024609

False

False

74

24

6

0.010312

False

False

75

25

25

0.992842

True

False

76

25

13

0.992827

False

False

77

25

33

0.992824

False

False

78

26

26

0.992835

True

False

79

26

38

0.992815

True

True

80

26

31

0.032934

False

False

81

27

27

0.992840

True

False

82

27

15

0.010981

False

False

83

27

5

0.010758

False

False

84

28

28

0.992839

True

False

85

28

21

0.032934

False

False

86

28

17

0.025500

False

False

87

29

29

0.992842

True

False

88

29

11

0.025500

False

False

89

29

31

0.010597

False

False

90

30

30

0.992834

True

False

91

30

3

0.025500

False

False

92

30

34

0.010597

False

False

93

31

31

0.992839

True

False

94

31

38

0.032934

False

False

95

31

26

0.032934

False

False

96

32

32

0.992842

True

False

97

32

20

0.992836

False

False

98

32

37

0.025500

False

False

99

33

33

0.992842

True

False

100

33

13

0.992828

False

False

101

33

25

0.992824

False

False

102

34

34

0.992836

True

False

103

34

17

0.010758

False

False

104

34

30

0.010597

False

False

105

35

36

0.992815

True

True

106

35

35

0.992815

True

False

107

35

18

0.992815

False

False