The current somatosensory selective attention based brain computer in terface (BCI) system has disadvantage of less command for multidegree deviceandlow information transmission rate. To solve these problems,a novel hybrid BCI system combing steadystate visual evoked potential (SSVEP) and somatosensory selective attention (SSA) is proposed in the paper. The SSVEP and event related desynchronization (ERD) can be elicited withtheaidof visual and somatosensory stimuli. In order to overcome the shortcomings of conventional feature extraction method which needs more heuristic knowledge, a deep learning algorithm is used to decode the EEG signal. In this method, the temporaldomainsignals of several channels are converted into temporalfrequencyspatial domain feature image. Eight subjects arerecruited to participate the experiment. The average accuracy of offline test is 8135%, which indicates that the proposed multimodal hybrid BCI based on SSVEP_SSA is feasible for instruction set extension and decoding precisely.