\documentclass[10pt]{article}

\usepackage{fullpage}
\usepackage{setspace}
\usepackage{parskip}
\usepackage{titlesec}
\usepackage[section]{placeins}
\usepackage{xcolor}
\usepackage{breakcites}
\usepackage{lineno}
\usepackage{hyphenat}





\PassOptionsToPackage{hyphens}{url}
\usepackage[colorlinks = true,
            linkcolor = blue,
            urlcolor  = blue,
            citecolor = blue,
            anchorcolor = blue]{hyperref}
\usepackage{etoolbox}
\makeatletter
\patchcmd\@combinedblfloats{\box\@outputbox}{\unvbox\@outputbox}{}{%
  \errmessage{\noexpand\@combinedblfloats could not be patched}%
}%
\makeatother


\usepackage[round]{natbib}
\let\cite\citep




\renewenvironment{abstract}
  {{\bfseries\noindent{\abstractname}\par\nobreak}\footnotesize}
  {\bigskip}

\titlespacing{\section}{0pt}{*3}{*1}
\titlespacing{\subsection}{0pt}{*2}{*0.5}
\titlespacing{\subsubsection}{0pt}{*1.5}{0pt}


\usepackage{authblk}


\usepackage{graphicx}
\usepackage[space]{grffile}
\usepackage{latexsym}
\usepackage{textcomp}
\usepackage{longtable}
\usepackage{tabulary}
\usepackage{booktabs,array,multirow}
\usepackage{amsfonts,amsmath,amssymb}
\providecommand\citet{\cite}
\providecommand\citep{\cite}
\providecommand\citealt{\cite}
% You can conditionalize code for latexml or normal latex using this.
\newif\iflatexml\latexmlfalse
\providecommand{\tightlist}{\setlength{\itemsep}{0pt}\setlength{\parskip}{0pt}}%

\AtBeginDocument{\DeclareGraphicsExtensions{.pdf,.PDF,.eps,.EPS,.png,.PNG,.tif,.TIF,.jpg,.JPG,.jpeg,.JPEG}}

\usepackage[utf8]{inputenc}
\usepackage[english]{babel}










\begin{document}

\title{Modeling orientation detection}



\author[1]{Hormet Yiltiz}%
\author[1]{Michael Landy}%
\affil[1]{New York University (NYU)}%


\vspace{-1em}



  
  \date{May 15, 2024}


\begingroup
\let\center\flushleft
\let\endcenter\endflushleft
\maketitle
\endgroup








\sloppy




Let $\theta_1, \theta_2 \ldots \theta_N \in [0, 2\pi)$ the preferred orientations of a network of $N$ neurons with circular Normal tuning curves and Poisson firing rates.

Model input is a target orientation $(\theta_t, c_t)$ superimposed on a mask orientation $(\theta_m, c_m)$, where a pair $(\theta, c)$ represents an orientation $\theta$ at contrast $c$. 

Tuning response of $i$th neuron to a set of orientations $(\theta_j, c_j)$: 

$$\lambda_i := \sum_j g \cdot c_j e^{2\kappa \cos(\theta_j)},$$ where $g$ is the gain of the network.

The Poisson response is a random sample from the Poisson distribution with $\lambda$ as the mean firing rate:

$$r_i \sim \mathrm{Pois}(\lambda_i)$$

The Poisson response acts as the input drive to divisive normalization, so we can normalized response:

$$R_i^{(t)} = \frac{\lfloor r_i \rfloor^2}{\sigma^2 + \mathbf{W}\mathbf{\lfloor r^{(t-1)} \rfloor^2}}$$

Then the weights are updated based on the Hebbian learning rule:

$$\mathbf{W}^{(t)} = \mathbf{W}^{(t-1)} + \alpha (\mathbf{R}\mathbf{R}^T - \mathbf{C})$$

where $\alpha$ is the Hebbian learning rate, $\mathbf{C}$ is the Hebbian learning homeostatis target.

This feedforward network runs until $t_{sat}$ such that network responses are stable. 

We initialize the network with initial weights $\mathbf{W}^{(0)}$ and a homeostatis target $\mathbf{C}$. The initial weights $\mathbf{W}^{(0)}$ is a uniform matrix scaled by the network's mean tuning response to gratings that are at full contrast and each neuron's preferred orientation. That is equivalent to the tuning response of a single neuron to a full-contrast grating at this neuron's preferred orientation. Thus:

$$\mathbf{W}^{(0)} := \frac{\mathbf{1}_{NN}}{\lambda_i(\theta = \theta_i, c=1)}$$

The homeostatis target is set to the network's mean response product over some period $t_0$ to $t_{max}$ to a set of orientations $\theta_1, \theta_2 \ldots \theta_k$ equally spaced within $[0, 2\pi)$, normalized by the size of the network:

$$\mathbf{C} := \bar{\mathbf{R}}(\theta_1, \theta_2 \ldots \theta_k) \bar{\mathbf{R}}(\theta_1, \theta_2 \ldots \theta_k)^T / N,$$

$$\bar{R_{ij}} := \bar{R_i}(\theta_j) = \frac{\sum_{t_0}^{t_{max}} R_i(\theta_j)}{t_{max}},$$

where $R_i(\theta_j)$ is the normalized (but not Hebbian weight learned) response of $i$th neuron to the orientation $\theta_j$ from the set of equally spaced orientations.
Furthermore, since the input to the time average of the normalized responses is Poisson process, the time average asymptotes to the Poisson rate $\lambda$. Thus, we could replace the time average by the asymptote for better precision and computational efficiency. That is:

$$R_{ij} := \lim_{t_{max} \to +\infty}\bar{R_{ij}} =  \lim_{t_{max} \to +\infty}\bar{R_i}(\theta_j) = \lim_{t_{max} \to +\infty} \frac{\sum_{t_0}^{t_{max}} R_i(\theta_j)}{t_{max}} = \lim_{t_{max} \to +\infty} \frac{\sum_{t_0}^{t_{max}} \frac{\lfloor r_i(\theta_j) \rfloor^2}{\sigma^2 + \mathbf{W}\mathbf{\lfloor r^{(t_{sat})} \rfloor^2}}}{t_{max}} = \frac{\lfloor \lambda_i(\theta_j) \rfloor^2}{\sigma^2 + \mathbf{W}\mathbf{\lfloor \lambda^{(t_{sat})} \rfloor^2}}$$

Here is a simple model readout that takes the normalized response of the neuron whose preferred orientation $\theta_i$ (pre-learning) matches target orientation $\theta_t$:

$$d := R_i, \theta_i = \theta_t$$

Alternatively, we could normalize that w.r.t. the total network response:

$$d := \frac{R_i}{\sum \mathbf{R}}, \theta_i = \theta_t$$

Or we can weight the network response with their pre-learning sensitivity to the target:

$$d := w \mathbf{R}, $$
$$ w_i := \lambda_i(\theta_t, c = 1)$$

\selectlanguage{english}
\FloatBarrier
\end{document}

