Logistic回归

学习的内容

• 1.Logistic回归案例实战

我的理解

• 利用Logistic回归进行分类的主要思想是：根据现有数据对分类边界线建立回归公式，以此进行分类。训练分类器的做法就是寻找最佳拟合参数，使用的是最优化算法，这里使用梯度下降法，因为输出结果是分类形式，所以用Sigmoid函数对结果进行处理。
  Sigmoid函数公式为：
  模型用一元函数：y = wx

  下面用Logistic回归原理和具体20个特征的数据集来预测马的存活问题。
  具体的代码实现如下：
  加载数据：代码略
  Sigmoid函数：

    def sigmoid(inX):
        return 1.0/(1+exp(-inX))

  随机梯度下降算法：
  这里采用随机形式是为了减少迭代数，加快模型收敛速度。

    def stocGradAscent1(dataMatrix, classLabels, numIter=150):
        m,n = shape(dataMatrix)
        weights = ones(n)
        for j in range(numIter):
            dataIndex = range(m)
            for i in range(m):
                # 每次迭代时调整学习率
                alpha = 4/(1.0+j+i)+0.0001
                # 样本数据索引随机获取
                randIndex = int(random.uniform(0,len(dataIndex)))
                # 梯度下降法更新模型参数
                h = sigmoid(sum(dataMatrix[randIndex]*weights))
                error = classLabels[randIndex] - h
                weights = weights + alpha * error * dataMatrix[randIndex]
                del(dataIndex[randIndex])
        return weights

  Logistic回归分类器：

    # 分类函数
    def classifyVector(inX, weights):
        prob = sigmoid(sum(inX*weights))
        if prob > 0.5: return 1.0
        else: return 0.0
    # 训练迭代的实现
    def colicTest():
        frTrain = open('horseColicTraining.txt')
        frTest = open('horseColicTest.txt')
        trainingSet = []; trainingLabels = []
        for line in frTrain.readlines():
            currLine = line.strip().split('\t')
            lineArr =[]
            for i in range(21):
                lineArr.append(float(currLine[i]))
            trainingSet.append(lineArr)
            trainingLabels.append(float(currLine[21]))
        trainWeights = stocGradAscent1(array(trainingSet), trainingLabels, 1000)
        errorCount = 0; numTestVec = 0.0
        for line in frTest.readlines():
            numTestVec += 1.0
            currLine = line.strip().split('\t')
            lineArr =[]
            for i in range(21):
                lineArr.append(float(currLine[i]))
            if int(classifyVector(array(lineArr), trainWeights))!= int(currLine[21]):
                errorCount += 1
        errorRate = (float(errorCount)/numTestVec)
        print "the error rate of this test is: %f" % errorRate
        return errorRate
    # 训练入口
    def multiTest():
        numTests = 10; errorSum=0.0
        for k in range(numTests):
            errorSum += colicTest()
        print "after %d iterations the average error rate is: %f" % (numTests, errorSum/float(numTests))

  运行结果：

    >>> import logRegres
    >>> logRegres.multiTest()
    logRegres.py:18: RuntimeWarning: overflow encountered in exp
      return 1.0/(1+exp(-inX))
    the error rate of this test is: 0.343284
    the error rate of this test is: 0.268657
    the error rate of this test is: 0.373134
    the error rate of this test is: 0.373134
    the error rate of this test is: 0.388060
    the error rate of this test is: 0.373134
    the error rate of this test is: 0.432836
    the error rate of this test is: 0.298507
    the error rate of this test is: 0.417910
    the error rate of this test is: 0.388060
    after 10 iterations the average error rate is: 0.365672
    >>>

  完整代码请查看：Logistic回归

意义

了解Logistic回归作分类器的原理及随机梯度下降的简单实现。