Schizophrenia-mimcking layer
Schizophrenia-mimicking layer is based on our study on nanometer-scale 3D structure of neuronal network in schizophrenia cases. The synchrotron raditation nano-CT analysis revealed that neurites of the anterior cingulate cortex are thin and tortuous in schizophrenia compared to control cases. Analysis of another area called temporal cortex delineated individual differences even between healthy controls. So we translated these findings into newly designed layers that mimic connection alteration in schizophrenia.
Test calculations using the schizophrenia layer indicated that 80% of weights can be eliminated without any changes in training procedures or network configuration. Very interestingly the schizophrenia layer completely suppresses overfitting and outperforms fully connected layer. Here is a typical example obtained using this python code with slight changes of num_epoch=200, idlist=[0.5, 0.0] and num_repeat=10. Over 60% of kernel weights of convolution layers can be zeroed with the same method without any accuracy loss. This study was published in Front Neurorobot.
How to implement schizophrenia-mimicking layer in your network
Our code runs on Tensorflow/Keras. Schizophrenia versions of fully connected layer and 2D convolution layer are available. Their usage is completely the same with official Keras layers, except for specifying parameter reduction ratio and its method. The default reduction ratio is 50%. A recommended (and default) reduction method is form='diagonal'.
- Download ‘schizo.py’ (Tensorflow 2.15) file from our repository to your working directory where your *.py file is placed. You can also find schizo.py files for older TF versions (Tensorflow 2.3 and 2.7). If you use them, download one of those versions compatible with your TF environment and rename the ‘schizo_tf2.x.py’ file to ‘schizo.py’.
- The following is an example code using a ‘SzDense’ layer in place of ‘Dense’ layer: ``` from tensorflow import keras from tensorflow.keras import layers import schizo
model = keras.Sequential([
layers.Flatten(),
# layers.Dense(512, activation=’relu’, kernel_initializer=’he_normal’),
schizo.SzDense(512, param_reduction=0.5, activation=’relu’, kernel_initializer=’he_normal’),
layers.Dense(num_class, activation=’softmax’)
])
```
In this example, the layers.Dense layer was commented out to replace it with a schizo.SzDense layer of 50% parameter reduction, which is defined with argument param_reduction. The reduction ratio best fit to your network depends on network configuration, but in most cases 50-70% seems to give good results. We recommend 50% as a first choice. No other modification of your python code is required.
Code used for preparing our paper figures
under construction.
Release notes
2024.3.30 Update: schizo.py compatible with TensorFlow 2.15
2022.3.28 Published in Front Neurorobot.
2022.1.22 Update: schizo.py compatible with TensorFlow 2.8
2020.10.3 Diagonal window of schizophrenia layer is now released.
2020.9.27 Revised accroding to official Keras implementations.
2020.9.24 Original release.
References
Mizutani et al (2022). Schizophrenia-mimicking layers outperform conventional neural network layers. Front. Neurorobot. 16, 851471. DOI
Mizutani et al (2021). Structural diverseness of neurons between brain areas and between cases. Transl. Psychiatry 11, 49.
DOI
pdf
Mizutani et al. (2020) Schizophrenia-mimicking layers outperform conventional neural network layers. arXiv
Mizutani et al. (2019) Three-dimensional alteration of neurites in schizophrenia. Transl Psychiatry 9, 85. nature.com