The use of filters is one of the major techniques for texture feature extraction. In this paper, we present approaches to filter design for optimal texture feature extraction. A generic feature extraction scheme of a finite impulse response filter and a local energy function is modelled, and the filter is optimized with respect to the Fisher criterion. We develop the necessary expressions for a gradient search optimization. Based on some assumptions, we also find a closed form solution to the problem. Then we extend the closed form solution to a more generalized unifying framework. Finally, the practical utilities of the filter design techniques are shown through experiments.