Aiming at the problem that the uncertainty of part processing position and feed direction of machine tool causes the change of the tool tip frequency response function (FRF), which leads to the uncertainty of cutting stability lobe diagram and chatterfree processing parameter prediction, a cutting stability prediction and optimization method is proposed combining the support vector regression (SVR) machine and genetic algorithm (GA). This method adopts the hammer impact modal test and spatial coordinate transformation to obtain the tool tip FRFs of different machining positions and feed directions in sample space; then combining the traditional cutting stability prediction method, a SVR prediction model of the limiting cutting depth is established, which takes the displacements of machine tool moving parts, the feed angle, spindle rotation speed, cutting width and the feed rate per tooth as the inputs; the SVR model is taken as the cutting stability constraint to establish the optimization model for the material removal rate (MRR); with the genetic algorithm (GA), the optimal configuration of the displacements of the moving axes, feed angle and cutting parameters is solved. A case study was performed on a certain machining center, and the experiment result shows that the obtained optimal configuration can achieve stable cutting, which verifies the effectiveness and feasibility of the proposed method.