By using generating functions technique, we investigate some properties of the k-ary Lyndon words. We give an explicit formula for the generating functions including not only combinatorial sums, but also hypergeometric function. We also derive higher-order differential equations and some formulas related to the k-ary Lyndon words. By applying these equations and formulas, we also derive some novel identities including the Stirling numbers of the second kind, the Apostol-Bernoulli numbers and combinatorial sums. Moreover, in order to compute numerical values of the higher-order derivative for the generating functions enumerating k-ary Lyndon words with prime number length, we construct an efficient algorithm. By applying this algorithm, we give some numerical values for these derivative equations for selected different prime numbers