In this paper, we introduce the Hermite-based poly-Bernoulli numbers and polynomials with q-parameter and give some of their basic properties including not only addition property, but also derivative properties and integral representations. We also define the Hermitebased $\lambda$-Stirling polynomials of the second kind and then provide some relations, identities of these polynomials related to the Stirling numbers of the second kind. We derive some symmetric identities for these families of special functions by applying the generating functions.