MA-E4xx シリーズ用のカーネルは、GitHub のリポジトリ で公開しています。

MA-E3xx と同じく git で管理されていますので、git パッケージの導入が必要です。
参考: [MA-E3xx] カーネルのビルド

[MA-E3xx] カーネルのビルド を参考に、リポジトリの clone, checkout まで済ませておきます。

kikuchi@develop:~/work$ git clone https://github.com/centurysys/linux-kernel-MAE.git
Cloning into 'linux-kernel-MAE'...
remote: Counting objects: 5857742, done.
remote: Compressing objects: 100% (3297/3297), done.
remote: Total 5857742 (delta 6079), reused 7155 (delta 5711), pack-reused 5848715
Receiving objects: 100% (5857742/5857742), 1.25 GiB | 8.48 MiB/s, done.
Resolving deltas: 100% (4945210/4945210), done.
Checking connectivity... done.
Checking out files: 100% (46095/46095), done.
kikuchi@develop:~/work$ cd linux-kernel-MAE/

kikuchi@develop:~/work/linux-kernel-MAE$ git checkout MA-E4xx/linux-4.4.y
Checking out files: 100% (39647/39647), done.
Branch MA-E4xx/linux-4.4.y set up to track remote branch MA-E4xx/linux-4.4.y from origin.
Switched to a new branch 'MA-E4xx/linux-4.4.y'
kikuchi@develop:~/work/linux-kernel-MAE$ 

デフォルトの config をロードします。

kikuchi@develop:~/work/linux-kernel-MAE$ make kumquat_MA_defconfig
  HOSTCC  scripts/basic/fixdep
  HOSTCC  scripts/kconfig/conf.o
  SHIPPED scripts/kconfig/zconf.tab.c
  SHIPPED scripts/kconfig/zconf.lex.c
  SHIPPED scripts/kconfig/zconf.hash.c
  HOSTCC  scripts/kconfig/zconf.tab.o
  HOSTLD  scripts/kconfig/conf
net/sched/Kconfig:44: warning: menuconfig statement without prompt
#
# configuration written to .config
#
kikuchi@develop:~/work/linux-kernel-MAE$ 

config がこのままでよければ、build を行います。

kikuchi@develop:~/work/linux-kernel-MAE$ make CROSS_COMPILE=arm-linux-gnueabihf- -j16
  CHK     include/config/kernel.release
  CHK     include/generated/uapi/linux/version.h
  CHK     include/generated/utsrelease.h
make[1]: 'include/generated/mach-types.h' is up to date.
  CHK     include/generated/timeconst.h
  CHK     include/generated/bounds.h
  CHK     include/generated/asm-offsets.h
  CALL    scripts/checksyscalls.sh
  CHK     include/generated/compile.h
... 中略 ...
  LD [M]  net/netfilter/xt_time.ko
  LD [M]  net/netfilter/xt_u32.ko
  LD [M]  net/xfrm/xfrm_ipcomp.ko
  LD [M]  sound/core/seq/snd-seq-midi.ko
  LD [M]  sound/core/snd-rawmidi.ko
  LD [M]  sound/usb/snd-usb-audio.ko
  LD [M]  sound/usb/snd-usbmidi-lib.ko
  AS      arch/arm/boot/compressed/piggy.xzkern.o
  LD      arch/arm/boot/compressed/vmlinux
  OBJCOPY arch/arm/boot/zImage
  Kernel: arch/arm/boot/zImage is ready
kikuchi@develop:~/work/linux-kernel-MAE$ 

uImage 形式のカーネルイメージを作成します。

kikuchi@develop:~/work/linux-kernel-MAE$ make CROSS_COMPILE=arm-linux-gnueabihf- -j16 uImage
  CHK     include/config/kernel.release
  CHK     include/generated/uapi/linux/version.h
  CHK     include/generated/utsrelease.h
make[1]: 'include/generated/mach-types.h' is up to date.
  CHK     include/generated/timeconst.h
  CHK     include/generated/bounds.h
  CHK     include/generated/asm-offsets.h
  CALL    scripts/checksyscalls.sh
  CHK     include/generated/compile.h
  CHK     kernel/config_data.h
  Kernel: arch/arm/boot/Image is ready
  Kernel: arch/arm/boot/Image is ready
  Kernel: arch/arm/boot/zImage is ready
  Kernel: arch/arm/boot/Image is ready
  Kernel: arch/arm/boot/zImage is ready
  UIMAGE  arch/arm/boot/uImage
Image Name:   Linux-4.4.30
Created:      Thu Mar 15 15:27:16 2018
Image Type:   ARM Linux Kernel Image (uncompressed)
Data Size:    3648072 Bytes = 3562.57 kB = 3.48 MB
Load Address: 42208000
Entry Point:  42208000
  Image arch/arm/boot/uImage is ready
kikuchi@develop:~/work/linux-kernel-MAE$