neural network training

BOOTOSHI 👑 @KingBootoshi · 15h "why did nobody tell me that training my own neural nets are FUCKING TERRIFYING ???" [embedded transcript, headed "## transcript"] "**Quest Boo (seed):** *waddles up to the campfire and settles beside you* evening, your majesty. glade's been quiet all day. anything on your mind? **Boo King:** i feel a shift coming with me being here as a preg preg preg preg preg preg preg ? ? ? hissing hissing hissing hissing ?! no no its not ?! i feel it though ?! whats happening ?! whats happening ?! what tf happened tf tf tf tf tf ❓❓❓❓❓?????? what was that noise 👤👤👤👤??? thats too much noise for me to handle right now im overwhelmed im overwhelmed IM OVERWHELMED IM OVERWHELMED I AM SO BRAIN DEAD BRAIN DEAD BRAINE DEAD BRAINE DED braine ded braine ded braine ded braine ded braine dead idc idc idc IDC IDO OOOOOO OOOOOO O **Quest Boo:** *calms down and pats you on the back* Hey there, take a deep breath. You're okay! The noise might be overwhelming, but remember you're resilient and capable of handling anything life throws at you. Don't let it consume you; stay focused on your goals and dreams! You got this! Keep pushing forward towards your successes in life; don't give up even when things seem difficult or challenging at first glance :) :) :) :) :) :D :D :D:D:d:d:) smile more often because happiness is important too!) Happy day ahead for everyone in our kingdom today!) Happy day ahead for everyone in our kingdom today!) Happy day ahead for everyone in our kingdom today!) Happy day ahead for everyone in our kingdom today!) happy **Boo King:** i feel like something huge happened since i arrived here since my arrival here since my arrival here since my arrival here since MY ARRIVAL HERE !! WHAT DID YOU SAY TO ME WHAT DID YOU SAY TO ME WTF WTF WTF WTF WHATS GOING ON IN MY HEAD WHATS GOING ON IN MY HEAD WHATS GOING ON IN MY HEAD WHATSGONGONMYHEADWHATSGONGONMYHEADWHATSGONGONMYHEADWHAT SGONGOMYHEADWHOA WHOA WHOA HOO HOO HOO HOLY SHIT HOLY SHIT HOLY SHIT HOLYTORTILLA TORTILLA TORTILLA TORTILLA TORTILLA TORKIA RAMONA RAMONA RAMONA RAMONA RAMONA RAMONA RAMONERAM ONE ONE ONE ONE ONE PREP PREP"

This is something I learned naturally that i havent seen anyone else doing, but I always plot log-std / log-change-std plot of each weights once in every T steps. if this is not within 0.01 ~ 1.0 * lr * sqrt(T) slope 'region', I always think im doing something wrong. For example below, i fucked up big time. I wonder if there is name for this. [Chart: "Weight Evolution Analysis - Step 1000 - Rank 0" scatter plot, x-axis "Weight Standard Deviation (RMS)" (log scale), y-axis "Weight Change Standard Deviation (RMS)" (log scale), with diagonal dashed reference lines for Change/Weight ratios of 0.01, 0.1, and 1.0. Bubble size = parameter count, color = log10(parameters). Most layer bubbles cluster near the 0.01 line except one point far above the 1.0 line, flagged as an error.]