上一篇文用C++手写了BP神经网络，事实上，生产环境中已经很少有人再亲自写神经网络了，已经出现了很多成熟的机器学习开源库。我安装了C++的shark库和python的pybrain库，在研究shark库的文档时发现shark库封装度太低了，用起来很麻烦，遂弃。虽然放弃C++的高效库很遗憾，但是后来发现pybrain库真是非常友好非常方便，于是分别写了分类器和函数拟合的例子及测评。
看两个库的源码时发现，shark库的机器学习模型比较多，pybrain的少一些，但是提供一些实时图形化的接口（虽然我嫌麻烦没用）。于是pybrain也可以与matlab和R语言争争份额了（虽然显然还是matlab和R比较专业）。
首先来看分类器的实现，还是那个寻找点与平面位置关系规则的问题，上代码，NNClassifier.py：
#!/usr/bin/env python
# -*- coding: utf-8 -*-
# 关于前馈神经网络二分类点集的尝试
# 因为是首次尝试使用pybrain，不使用快捷方式创建神经网络以求思路清晰。
__author__ = 'Bipedal Bit, hmemoryl@163.com'
from pybrain.structure import FeedForwardNetwork
from pybrain.structure import LinearLayer, SigmoidLayer
from pybrain.structure import FullConnection
from pybrain.datasets import SupervisedDataSet
import random
from pybrain.supervised.trainers import BackpropTrainer
import time
from pybrain.tools.customxml.networkwriter import NetworkWriter
from pybrain.tools.customxml.networkreader import NetworkReader
# ================== 构造神经网络 ======================
# 建立前馈神经网络FNN
FNN = FeedForwardNetwork()
# 逐层建立神经元
# 建立m个输入参数的输入层（inLayer）
inLayer = LinearLayer(7, name = 'inLayer')
# 建立隐含层
hiddenLayer0 = SigmoidLayer(30, name = 'hiddenLayer0')
# 建立n个输出参数的输出层（outLayer）
outLayer = SigmoidLayer(2, name = 'outLayer')
# 将层加入神经网络（向神经网络内添加不同功能的神经元）
FNN.addInputModule(inLayer)
FNN.addModule(hiddenLayer0)
FNN.addOutputModule(outLayer)
# 建立层之间的连接，使用全连接，即两层的神经元之间两两全部连接
conn_inToHidden0 = FullConnection(inLayer, hiddenLayer0)
conn_hidden0ToOut = FullConnection(hiddenLayer0, outLayer)
# 将连接加入神经网络
FNN.addConnection(conn_inToHidden0)
FNN.addConnection(conn_hidden0ToOut)
# 神经元拓扑排序，准备使用
FNN.sortModules()
# ================== 构造数据集 ======================
# 建立带监督的数据集，参数为输入维数和输出维数
DS = SupervisedDataSet(7, 2)
# 向数据集添加样本
# 将一个0~1的随机数扩展到适当的范围
def expand(n):
return n*20 - 10
# 数据组数
dataCnt = 2000
for i in xrange(1, dataCnt):
# 生成归一化样本数据
_x = random.random()
_y = random.random()
_z = random.random()
_a = random.random()
_b = random.random()
_c = random.random()
_d = random.random()
# 扩散出样本真实值
x = expand(_x)
y = expand(_y)
z = expand(_z)
a = expand(_a)
b = expand(_b)
c = expand(_c)
d = expand(_d)
# 计算监督值
above = 1 if z > (a*x + b*y + d)/(-c) else 0
below = 1 if above == 0 else 0
# 添加数据
DS.addSample([_x, _y, _z, _a, _b, _c, _d], [above, below])
# 分别索引输入/输出数据集合
X, Y = DS['input'], DS['target']
# 按比例切分训练集与测试集（训练集：测试集 = 8:2）
DSTrain, DSTest = DS.splitWithProportion(0.8)
# 分别索引训练集和测试集的输入/输出数据集合
XTrain, YTrain = DSTrain['input'], DSTrain['target']
XTest, YTest = DSTest['input'], DSTest['target']
# =================== 读取神经网络 =====================
FNN = NetworkReader.readFrom('networkClasssifier.xml')
# ================== 训练神经网络 ======================
# 使用BP算法训练神经网络
trainer = BackpropTrainer(FNN, DSTrain, verbose = True, learningrate = 0.01, lrdecay = 1, momentum = 0)
# 开始计时
t_start = time.clock()
# 训练至收敛
trainer.trainUntilConvergence(maxEpochs = 100)
# 计时结束
t_end = time.clock()
# 输出训练用时
print '训练用时：', t_end - t_start, 's'
# =================== 存储神经网络 =====================
NetworkWriter.writeToFile(FNN, 'networkClasssifier.xml')
# =================== 结果可视化 =======================
# ====================== 测试 ==========================
successCnt = 0
for i in xrange(0, len(XTest)-1):
# 获取测试数据
x = XTest[i]
y = YTest[i]
# 测试
prediction = FNN.activate(x)
if (y[0] - y[1])*(prediction[0] - prediction[1]) > 0:
successCnt += 1
print '测试：', len(XTest)
print '命中：', successCnt
print '命中率：', (successCnt*1.)/len(XTest)*100, '%'
训练结果我不太意外，误差函数值在0.05左右，算不上好，正确率在82%左右，只能说还行。这次只是简单测评，就不对收敛程度做太高要求了。（其实是我把隐含层神经元都加到30个了精度也不是很高，有点累了，毕竟经验公式说这种输入输出参数量最多加到13个）
最后的神经网络保存文档（权重数组），networkClassifier.xml：
<?xml version="1.0" ?>
<PyBrain>
<Network class="pybrain.structure.networks.feedforward.FeedForwardNetwork" name="FeedForwardNetwork-5">
<name val="u'FeedForwardNetwork-5'"/>
<Modules>
<LinearLayer class="pybrain.structure.modules.linearlayer.LinearLayer" inmodule="True" name="inLayer">
<dim val="7"/>
<name val="'inLayer'"/>
</LinearLayer>
<SigmoidLayer class="pybrain.structure.modules.sigmoidlayer.SigmoidLayer" name="outLayer" outmodule="True">
<dim val="2"/>
<name val="'outLayer'"/>
</SigmoidLayer>
<SigmoidLayer class="pybrain.structure.modules.sigmoidlayer.SigmoidLayer" name="hiddenLayer0">
<dim val="30"/>
<name val="'hiddenLayer0'"/>
</SigmoidLayer>
</Modules>
<Connections>
<FullConnection class="pybrain.structure.connections.full.FullConnection" name="FullConnection-3">
<inmod val="inLayer"/>
<outmod val="hiddenLayer0"/>
<Parameters>[-4.4634646760183774, -1.8663551824786604, 1.5796984784944188, -4.1545734140399917, -1.4235379909335284, 10.814418886792573, -0.35697065520281313, 3.7168599840362613, -1.3954006078608174, 3.2888708257680479, -1.2728713382523704, 5.1682485849770519, -10.345459393449577, 0.40944729400489821, -0.86432327459917391, 0.87415267739043501, -1.1261847464737353, 0.26550824904566483, -0.69655632472406404, -1.0926496454783619, -1.3241014363212174, -1.2345262671004642, -5.3545154953074556, -3.2977309106602868, -1.9192142648212187, 3.5219032402205235, 8.8243116475870895, -0.34711073857964059, -0.76652402450096546, -0.45800625182882904, 0.10921234310650819, 1.0906882881030282, 0.58122385851806679, 0.74902769546891046, 0.57778560396382117, -0.36268964602939496, -1.4560716552974731, 0.025807475393432977, -0.52324632490359346, -1.4768259974689912, -1.3226548504200857, -0.56024431774602601, 0.5253813385807985, -2.3601368528359932, 3.4391924327008545, -3.4529059457340887, -0.36798304125763204, 0.24642387811765787, 0.74392363594535504, -1.4200092817628436, -0.95882940181717424, 0.90204884683974906, -2.2123307425066581, -1.2927495594815508, -2.6418298086483243, 0.0089737728359628229, -0.96131150512314989, -0.42386913703414159, 2.7423827971590629, 0.49257080970361217, -0.73140003600797654, -0.97237673817617065, 0.021672107067701795, -0.30730995794768651, -0.28377110259164756, -1.3769421502868895, -0.49923480994397734, 0.63167130189837051, 0.72750574299622528, -0.50823243724360356, -2.3249236293984685, -1.0084930398088603, 0.55483423319351144, -3.0895951468032683, -1.569810215716863, -4.6712778120909002, 2.1476977927094589, -0.92601922552049853, 0.80913122603871213, 1.0167819396610751, 1.2578916110729925, 1.1336741124853456, -1.4462571148912973, 0.4498637409091461, -0.0046570255381198191, 1.0130490743040022, 0.067808742574706565, -3.1665123334368257, -1.3695841902759027, -0.61944129404109394, 0.32712520616674406, -4.0944887153367588, 0.33425112596763401, 0.53786026328450398, 3.783707069693385, -0.73771116333542275, -7.3116843863108363, -0.90847319043398045, -1.3072997563815252, 1.5864357535576585, -2.1773369395271285, 0.16582713303194607, 0.62423604397793453, 0.59389298905810417, -0.31484499272512129, 5.0708486445028633, 0.91681404432938907, 0.31193631582711534, -5.4473204154043851, -0.7847240929315491, -7.6951030649320984, -0.32014838444510901, 0.56829986867023885, -0.29653458065896326, 0.51800388373753137, -0.56704382593349767, -0.25264606109631066, 0.78514270624403915, 0.67787739064705022, -0.0014430992506962728, -1.1915649958214867, -2.8339864138165369, -1.2920958546780335, -1.2004239162539645, -5.5518360974893373, 2.6887049948956299, -1.8837019633804577, 2.8587109449831924, 3.8684384835516568, -2.7090025688862394, -3.3154340336962216, 1.0688334874085363, 0.43275994294040698, -3.8784776661421345, -0.8578678330233066, 1.8270738585600814, 5.4477321487215189, 2.7554123218891138, -6.9965304625370077, 0.91986888975642622, 1.6206020126465939, 0.40565735648759632, 1.6264383151366206, -2.0910948725615417, -0.96054739666906863, 0.83967430154853273, 0.17826444997023716, 0.93718547827806598, -5.2232302716871377, -0.37715953425266679, 1.1057273981735851, -5.0228630028488084, 8.7833592718428406, -0.17862770462286087, 0.071119534208408983, 0.61781354218205642, -2.0482041559354802, -1.2286398440013546, 0.30582092080786355, 0.74602342033773583, -0.12623275590092933, -0.89369108883986548, -1.0173825208087091, 0.21764568858325722, 1.1182634120634272, 0.44364418276133, 1.138196191040812, 0.10662608395776786, -0.25838212820199019, -0.86982182253008533, -0.55680951312883542, 0.25307613718779792, -0.90155685220035831, 1.835084251778919, -0.81174579278015824, 0.48215346567645934, 6.267187930149162, -2.1805487376071957, 1.0628647712248505, -3.0943161262836401, -4.9327587898336835, 1.3749005504013798, -5.5102212316356844, -0.96863134222439218, 0.8268862262896568, 4.3540259268851402, -0.50518156739373588, 3.8934730549155523, -0.39568193061006912, -3.7859120847720105, 2.0595665943684325, -2.6633982082854342, -4.0807360028025208, 3.7442177075679623, 5.11773145948146, -0.44630616509537863, 0.20714091867709217, -0.6873175410894995, 0.061460095180831932, -0.44453400594106057, 0.35250696385984698, 2.2565455035241082, -0.079401700376001166, -0.18177974449629683, 0.71736053566879954, 0.54270386006022042, 0.75951868740937811, 1.3078378324277897, -2.1881100701125251, 0.84052630355979829]</Parameters>
</FullConnection>
<FullConnection class="pybrain.structure.connections.full.FullConnection" name="FullConnection-4">
<inmod val="hiddenLayer0"/>
<outmod val="outLayer"/>
<Parameters>[-8.2129041568200112, -6.7995955920941888, 0.48890334859207907, 7.6066781773027499, 2.8270407553317769, -1.0008489151414828, 3.0303929667267862, -2.4507639356974593, 1.9205137372795547, -2.0677111079872867, -4.6278733853833467, 3.3159637261922645, 0.4984027688395733, 4.7415120162351183, -2.5033978652108386, 5.1877234768369922, 0.87623718221836022, -5.0415283550332814, 4.0707627481245554, -6.4411441580882105, 2.6369969676415286, -8.5775684178164635, -1.313121167107631, -0.89764250697180914, -0.78729432668282306, -4.1300664676586489, 5.6446245751676667, -4.5102393183383498, 0.1106246177522491, 3.2716628012569964, 8.2844165753556958, 6.9238002822405154, 0.33656468267775197, -7.6960641823743456, 0.84148765790877422, 1.7483968344478173, -2.9938885121262571, 2.2508496743698845, -3.3237973590853258, 0.25502346696281558, 4.4894675338875496, -2.5350467550996507, -0.11269663125646072, -4.7593000820253231, 1.8099587344845871, -5.1168105240606856, -1.7100116403752497, 4.8179029808866174, -4.1421556373685728, 6.5870613347344591, -1.9261388149076637, 8.4927720262891935, 1.6203591174629115, -1.702384877536026, 1.9822996827269457, 3.6477517464649072, -5.4008040115101963, 4.720080872228464, -0.76546727507485091, -3.6703618616272795]</Parameters>
</FullConnection>
</Connections>
</Network>
</PyBrain>
函数拟合实验选择了简单的a+b问题，上代码，NNRegression.py：
#!/usr/bin/env python
# -*- coding: utf-8 -*-
# 关于前馈神经网络函数拟合的尝试
# 因为是首次尝试使用pybrain，不使用快捷方式创建神经网络以求思路清晰。
__author__ = 'Bipedal Bit, hmemoryl@163.com'
from pybrain.structure import FeedForwardNetwork
from pybrain.structure import LinearLayer, SigmoidLayer
from pybrain.structure import FullConnection
from pybrain.datasets import SupervisedDataSet
import random
from pybrain.supervised.trainers import BackpropTrainer
import time
from pybrain.tools.customxml.networkwriter import NetworkWriter
from pybrain.tools.customxml.networkreader import NetworkReader
# ================== 构造神经网络 ======================
# 建立前馈神经网络FNN
FNN = FeedForwardNetwork()
# 逐层建立神经元
# 建立m个输入参数的输入层（inLayer）
inLayer = LinearLayer(2, name = 'inLayer')
# 建立隐含层
hiddenLayer0 = SigmoidLayer(20, name = 'hiddenLayer0')
# 建立n个输出参数的输出层（outLayer）
outLayer = LinearLayer(1, name = 'outLayer')
# 将层加入神经网络（向神经网络内添加不同功能的神经元）
FNN.addInputModule(inLayer)
FNN.addModule(hiddenLayer0)
FNN.addOutputModule(outLayer)
# 建立层之间的连接，使用全连接，即两层的神经元之间两两全部连接
conn_inToHidden0 = FullConnection(inLayer, hiddenLayer0)
conn_hidden0ToOut = FullConnection(hiddenLayer0, outLayer)
# 将连接加入神经网络
FNN.addConnection(conn_inToHidden0)
FNN.addConnection(conn_hidden0ToOut)
# 神经元拓扑排序，准备使用
FNN.sortModules()
# ================== 构造数据集 ======================
# 建立带监督的数据集，参数为输入维数和输出维数
DS = SupervisedDataSet(2, 1)
# 向数据集添加样本
# 数据组数
dataCnt = 1000
for i in xrange(1, dataCnt):
# 生成归一化样本数据
a = random.random()
b = random.random()
# 计算监督值
s = a*20 - 10 + b*20 - 10
# 添加数据
DS.addSample([a, b], [s])
# 分别索引输入/输出数据集合
X, Y = DS['input'], DS['target']
# 按比例切分训练集与测试集（训练集：测试集 = 8:2）
DSTrain, DSTest = DS.splitWithProportion(0.8)
# 分别索引训练集和测试集的输入/输出数据集合
XTrain, YTrain = DSTrain['input'], DSTrain['target']
XTest, YTest = DSTest['input'], DSTest['target']
# =================== 读取神经网络 =====================
FNN = NetworkReader.readFrom('networkRegression.xml')
# ================== 训练神经网络 ======================
# 使用BP算法训练神经网络
trainer = BackpropTrainer(FNN, DSTrain, verbose = True, learningrate = 0.01, lrdecay = 1, momentum = 0)
# 开始计时
t_start = time.clock()
# 训练至收敛
trainer.trainUntilConvergence(maxEpochs = 100)
# 计时结束
t_end = time.clock()
# 输出训练用时
print '训练用时：', t_end - t_start, 's'
# =================== 存储神经网络 =====================
NetworkWriter.writeToFile(FNN, 'networkRegression.xml')
# =================== 结果可视化 =======================
# ====================== 测试 ==========================
for i in xrange(0, 10):#len(XTest)-1):
# 获取测试数据
x = XTest[i]
y = YTest[i]
# 测试
prediction = FNN.activate(x)
print '期望值：', y
print '预测值：', prediction
print '--------------------------------------'
评测结果有些诡异，尽管我丧心病狂的把隐含层神经元数慢慢加到了20个之多，收敛精度依然不太高。误差函数值在0.016左右，计算结果多的时候能有一点几的误差。至于为什么诡异，你们知道我用C++手写的那个BP神经网络a+b拟合测评结果什么样吗？我每次增加隐含层神经元，精度都在提高，最后加到11个神经元时，误差值只有10^-7的数量级了。我在想，莫非pybrain的BP实现写的比我还糟糕？当然我没有闲到对比源码找差别的地步，毕竟这是在扩展知识面而不是开题写论文，要学的东西还有很多。
训练结果，networkRegression.xml：
<?xml version="1.0" ?>
<PyBrain>
<Network class="pybrain.structure.networks.feedforward.FeedForwardNetwork" name="FeedForwardNetwork-5">
<name val="u'FeedForwardNetwork-5'"/>
<Modules>
<LinearLayer class="pybrain.structure.modules.linearlayer.LinearLayer" inmodule="True" name="inLayer">
<dim val="2"/>
<name val="'inLayer'"/>
</LinearLayer>
<LinearLayer class="pybrain.structure.modules.linearlayer.LinearLayer" name="outLayer" outmodule="True">
<dim val="1"/>
<name val="'outLayer'"/>
</LinearLayer>
<SigmoidLayer class="pybrain.structure.modules.sigmoidlayer.SigmoidLayer" name="hiddenLayer0">
<dim val="20"/>
<name val="'hiddenLayer0'"/>
</SigmoidLayer>
</Modules>
<Connections>
<FullConnection class="pybrain.structure.connections.full.FullConnection" name="FullConnection-4">
<inmod val="inLayer"/>
<outmod val="hiddenLayer0"/>
<Parameters>[0.66989695089493184, 0.20154541787085151, -0.50128908433773467, -0.37527744480307418, -0.74611014578997958, -0.12337190284043228, -0.26139108724410959, -0.62223790666112244, 0.76934737325151314, 0.097273365366757986, -0.70241911419677627, -0.1675521617762078, -0.52875791799833338, -0.34701820533745775, 0.91136351597023102, -0.051368168009540249, -0.46415381076169848, -0.41344842847904884, -0.3637989937993043, -0.51709189928560939, 0.021917540062381426, -0.91373858562576571, 0.18558709721271155, 0.70146289478301882, 0.1657642703601043, 0.72042804496432955, -0.051995836002024759, -0.83682492233986472, -0.57633470846976453, -0.29793296256418189, -0.0042668700361729222, 0.89613628191472339, 0.78248084199047585, 0.083752407601260875, -0.34426021947865065, -0.53677318854588607, -0.59630570148683881, -0.2772033711210245, -0.29072335479904238, -0.59221753051512371]</Parameters>
</FullConnection>
<FullConnection class="pybrain.structure.connections.full.FullConnection" name="FullConnection-3">
<inmod val="hiddenLayer0"/>
<outmod val="outLayer"/>
<Parameters>[7.0033787939978653, -8.9204704009603475, -7.0019357487260141, -9.1851348838825597, 11.222280489358978, -9.1321512616209173, -10.187840330915042, 12.40878268578923, -8.1946287348945397, -9.729867458003703, -10.577209749285847, 12.709727310110253, 9.5182614987221701, -8.4580456500212975, -7.9531433608618309, 11.918092172946009, 10.86011334049692, -8.310811818166254, -9.5385851767871586, -9.531838664642386]</Parameters>
</FullConnection>
</Connections>
</Network>
</PyBrain>
以上。欢迎留言探讨。
