Accompanying advancement of brain imaging technologies, which can monitor neural activities in the entire brain, the significance of analytical methods for finding important principles of information processing and network structures in a vast amount of nerve information data is increasing. Visual analytics is a powerful analytical tool as it converts analytical results instantaneously into an interpretable image and with its conversational interface, is easy to try various analyses of brain structure and activity.
The brain actualizes advanced information processing by a huge number of interconnecting nerve cells forming a network and dynamically transmitting signals by neural firing. The cerebral cortex is classified into parts (areas), and nerve cells in the same area are known to show similar activity characteristics. Methods have recently advanced for identifying a bundle of nerve fibers and measuring neural activity in the brain without harming the brain tissue. For example, diffusion magnetic resonance imaging and probabilistic tractography is based on magnetic resonance and can visualize the neural network in the brain by detecting a bundle of nerve fibers. On the other hand, functional magnetic resonance imaging (fMRI) can visualize the level of neural activity in the brain with high spatial resolution by measuring changes in blood flow and oxygen consumption accompanying neural activity.
To elucidate the dynamics of information processing in the brain, it is necessary to identify the direction of neural information transmission in the neural network and clarify the effects (i.e., the causal relationship) of neural activity in one area on neural activity in another area. However, MRI measurement is incapable of identifying the direction of a nerve-fiber bundle (direction of neural information transmission), and using fMRI is difficult when comparing the neural activity dynamics in two different areas because the time resolution is low (order of seconds). Thus, I have also used magnetoencephalography (MEG) and electroencephalogram (EEG) besides MRI and fMRI measurements in my research to elucidate the dynamics of cerebral nerve activity.
To investigate neural activity dynamics involved in the processing of visual perception information, I proposed an integrated analysis performing both fMRI and MEG during a visual perception task (Fig. 1) and processed the data of the two analyses. In the proposed method, the area in the brain where neural activity occurs is identified from fMRI data, and spatial restrictions are imposed to the solution when neural activity is determined from MEG data. By applying the scheme of the generalized least squares method under the spatial restrictions, it is possible to reconstruct neural activity in the target area with high time resolution (order of milliseconds). In order to further clarify the associativity among areas, it will be necessary to select best methods among available associative analysis methods and perform multilateral analysis by supplementing necessary information.
Visual analytics, which has an easy-to-use conversational interface and facilitates understanding by graphical display of analytical methods and results, is an excellent tool for gleaning useful information from a complicated and vast amount of data. Until now, visual analytical methods targeting fMRI data have been developed and used to determine the area of the brain to focus (region of interest: ROI) in analysis. However, research developing a visual analytical method that can integrate and analyze brain data measured using a plural number of methods such as MRI, fMRI, and MEG is still under way. Therefore, to clarify the dynamics of brain information processing, I am conducting research and development of a new method of visual analysis that can supplementarily use data obtained from two or more measuring methods.
For example, various methods have been proposed for investigating associativity among brain areas using historical (timing) data such as those measured by MEG and EEG. Such methods include Granger causality (based on a multivariate autoregressive model), dynamic causal modeling (based on a signal generation model), and recently proposed convergent cross-mapping which is based on nonlinear state space reconstruction. To investigate associativity between brain areas, it is necessary to not only select which of these methods is to be used but also to determine information necessary for analysis (such as information of ROI and preliminary parameters). Each method of associativity analysis should not be exclusive; but in some cases of detailed analysis, it needs to be combined with another method. In order to respond to these demands, I am developing a visual analytic environment in which brain measurement data from another method such as fMRI can be used to supplement necessary information (Fig. 2(b)). Fig. 2(c) is an example of cross-region associativity analyzed by visual analysis that uses dynamical causal modeling which is under development. By incorporating graph drawing anew, it is easy to compare associativity results between models. It is now demanded to expand multiple analytical methods so they can be handled on a single platform.
In the future, I am planning to apply the proposed integrated visual analysis method for supporting diagnosis at medical settings. Diseases that accompany declines in behavioral or/and cognitive function may be manifested as changes in dynamic properties of cerebral nerve activity. The integrated visual analysis method will contribute to early detection of the disease and thus contribute to improvement of medicine and rehabilitation.