Parameter Learning

Parameter learning of a BBN proceeds after learning the structure. Let’s see how this works.

Load data

Let’s read our data into a Spark DataFrame SDF.

from pysparkbbn.discrete.data import DiscreteData

sdf = spark.read.csv('hdfs://localhost/data-1479668986461.csv', header=True)
data = DiscreteData(sdf)

Structure learning

Let’s pick the MWST algorithm to learn the structure.

from pysparkbbn.discrete.scblearn import Mwst

mwst = Mwst(data)
g = mwst.get_network()

Parameter learning

After we have a structure, we can learn the parameters.

from pysparkbbn.discrete.plearn import ParamLearner
import json

param_learner = ParamLearner(data, g)
params = param_learner.get_params()

print(json.dumps(params, indent=2))

{
  "n4": [
    0.40255,
    0.59745
  ],
  "n3": [
    0.9914285714285714,
    0.008571428571428572,
    0.3994593202883625,
    0.6005406797116375,
    0.39842913245269546,
    0.6015708675473045,
    0.010762975364745277,
    0.9892370246352548
  ],
  "n2": [
    0.7991074402184773,
    0.20089255978152268,
    0.20473230399037498,
    0.795267696009625
  ],
  "n5": [
    0.2997143212023351,
    0.5976897279841014,
    0.10259595081356353,
    0.29324629676123526,
    0.09649343041258683,
    0.6102602728261779
  ],
  "n1": [
    0.75065,
    0.24935
  ]
}