18-10-25

Author: vignesh14052002

knowledge_base/AI/Evaluation.md

@@ -0,0 +1,16 @@
+identification problem
+    - F1 score
+
+translation problem
+    - BLEU
+    - CODEBLUE - https://arxiv.org/pdf/2009.10297
+    - BERT
+    - CODEBERT
+
+Papers to read
+- https://arxiv.org/pdf/2301.09043
+- https://arxiv.org/pdf/2408.16498
+- https://arxiv.org/pdf/1906.04903
+- https://arxiv.org/pdf/2501.12934
+
+

knowledge_base/AI/Observability.md

@@ -0,0 +1,5 @@
+# Observability
+Collect useful information from application for debugging or app insight purpose
+
+[FastAPI Observability Setup](https://github.com/blueswen/fastapi-observability)
+- will prometheus, loki, tempo uses db to store data?

knowledge_base/AI/ThinkPad.md

@@ -88,4 +88,5 @@
  - not much useful
 
 # explore
-- anthropic prompt improver and prompt evaluator
+- anthropic prompt improver and prompt evaluator
+- neo4j certification

knowledge_base/AI/Transformers.md

@@ -0,0 +1,21 @@
+
+## Understanding Transformers
+
+Softmax
+ - Normalizing values so that adding them will result 1 and they will represent probablity
+ - Each value is raised to e (ie e^x) to convert negatives and divided by the total
+
+Temperature
+ - the `t` variable in softmax function `e^(x/t)`
+ - the higher the `t`, the smoother the distribution, so less related token will be sampled
+
+
+ ### Defeating Non determinism in LLMs
+ [Blog](https://thinkingmachines.ai/blog/defeating-nondeterminism-in-llm-inference/)
+ - What is non determinism?
+    - If same prompt is given to LLM multiple times, all the responses will not be exactly similar
+ - Why it happens?
+    - Output Sampling - temperature parameter
+      - Set to 0 to avoid it
+    - concurrent operations on floating point numbers
+      - Make each operation batch invariant