generated from badjware/jupiter-notebook-template
213 lines
26 KiB
Plaintext
213 lines
26 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"This notebook is an exploration of how a RAG could be created. The idea is to create a chatbot who can answer questions based on the documents it has been trained on.\n",
|
|
"\n",
|
|
"Requirements are as follow:\n",
|
|
"* Source documents are in markdown format\n",
|
|
"* Source documents are stored in a git repository\n",
|
|
"* Everything needs to be self-hosted\n",
|
|
" * An Ollama server is already running locally (https://localhost:11434)\n",
|
|
"* The interface is unimportant for now\n",
|
|
" * Eventually, we want it to be a bot hosted in Teams and/or Discord"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Steps\n",
|
|
"\n",
|
|
"### Step 1: Fetch the documents from the git repository\n",
|
|
"\n",
|
|
"We will use `gitpython` to clone the repository and fetch the documents.\n",
|
|
"\n",
|
|
"For this notebook, we will ingest the documentation of prometheus-operator located in this repository: https://github.com/prometheus-operator/prometheus-operator.git. The documentation is within the *Documentation/* folder of this repository.\n",
|
|
"\n",
|
|
"First thing first, we need to clone the repository. If it already exists locally, we can just pull the latest changes."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 1,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"import os\n",
|
|
"\n",
|
|
"from git import Repo\n",
|
|
"\n",
|
|
"repo_url = \"https://github.com/prometheus-operator/prometheus-operator.git\"\n",
|
|
"local_repo_path = \"./prometheus-operator\"\n",
|
|
"\n",
|
|
"if not os.path.exists(local_repo_path):\n",
|
|
" Repo.clone_from(repo_url, local_repo_path)\n",
|
|
"else:\n",
|
|
" repo = Repo(local_repo_path)\n",
|
|
" repo.remotes.origin.pull()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"Now that we have a local copy of the repository, we can find the documents we are interested in within and list them."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 2,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Found 40 documents.\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"documentation_root = os.path.join(local_repo_path, \"Documentation\")\n",
|
|
"documentation_files = []\n",
|
|
"\n",
|
|
"# Walk through the directory and find all markdown files\n",
|
|
"for root, dirs, files in os.walk(documentation_root):\n",
|
|
" for file in files:\n",
|
|
" if file.endswith(\".md\"):\n",
|
|
" documentation_files.append(os.path.join(root, file))\n",
|
|
"\n",
|
|
"print(f\"Found {len(documentation_files)} documents.\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Step 2: Ingest the documents in a vector database\n",
|
|
"\n",
|
|
"To build our RAG, we need to store the documents in a vector database. Several options are available:\n",
|
|
"* [FAISS](https://faiss.ai/)\n",
|
|
"* [ChromaDB](https://www.trychroma.com/)\n",
|
|
"* [Qdrant](https://qdrant.tech/)\n",
|
|
"* etc.\n",
|
|
"\n",
|
|
"For this example, we will use ChromaDB because it is easy to set up and use. Helpfully, ChromaDB is able to automatically generate embeddings for us. We will store the documents in a collection called `documentation`. The collection will live in-memory, but in a more complete setup, we could setup Chroma in an client-server mode and/or with persistence enabled."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 3,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"import chromadb\n",
|
|
"\n",
|
|
"chroma_client = chromadb.Client()\n",
|
|
"collection = chroma_client.create_collection(name=\"documentation\")\n",
|
|
"\n",
|
|
"# Read the contents of each document and store them in a list\n",
|
|
"documents = []\n",
|
|
"for file in documentation_files:\n",
|
|
" with open(file, \"r\") as f:\n",
|
|
" content = f.read()\n",
|
|
" documents.append(content)\n",
|
|
"\n",
|
|
"# Add the documents to the collection\n",
|
|
"collection.add(documents=documents, ids=documentation_files)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"Now that we have built our collection, we can try to query it. Let's search for a document about prometheus."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 8,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"{\n",
|
|
" \"ids\": [\n",
|
|
" [\n",
|
|
" \"./prometheus-operator/Documentation/platform/prometheus-agent.md\",\n",
|
|
" \"./prometheus-operator/Documentation/proposals/202201-prometheus-agent.md\",\n",
|
|
" \"./prometheus-operator/Documentation/additional-scrape-config.md\"\n",
|
|
" ]\n",
|
|
" ],\n",
|
|
" \"embeddings\": null,\n",
|
|
" \"documents\": [\n",
|
|
" [\n",
|
|
" \"---\\nweight: 204\\ntoc: true\\ntitle: Prometheus Agent\\nmenu:\\n docs:\\n parent: user-guides\\nlead: \\\"\\\"\\nimages: []\\ndraft: false\\ndescription: Guide for running Prometheus in Agent mode\\n---\\n\\n{{< alert icon=\\\"\\ud83d\\udc49\\\" text=\\\"Prometheus Operator >= v0.64.0 is required.\\\"/>}}\\n\\nAs mentioned in [Prometheus's blog](https://prometheus.io/blog/2021/11/16/agent/), Prometheus Agent\\nis a deployment model optimized for environments where all collected data is forwarded to\\na long-term storage solution, e.g. Cortex, Thanos or Prometheus, that do not need storage or rule evaluation.\\n\\nFirst of all, make sure that the PrometheusAgent CRD is installed in the cluster and that the operator has the proper RBAC permissions to reconcile the PrometheusAgent resources.\\n\\n```yaml mdox-exec=\\\"cat example/rbac/prometheus-operator/prometheus-operator-cluster-role.yaml\\\"\\napiVersion: rbac.authorization.k8s.io/v1\\nkind: ClusterRole\\nmetadata:\\n labels:\\n app.kubernetes.io/component: controller\\n app.kubernetes.io/name: prometheus-operator\\n app.kubernetes.io/version: 0.80.0\\n name: prometheus-operator\\nrules:\\n- apiGroups:\\n - monitoring.coreos.com\\n resources:\\n - alertmanagers\\n - alertmanagers/finalizers\\n - alertmanagers/status\\n - alertmanagerconfigs\\n - prometheuses\\n - prometheuses/finalizers\\n - prometheuses/status\\n - prometheusagents\\n - prometheusagents/finalizers\\n - prometheusagents/status\\n - thanosrulers\\n - thanosrulers/finalizers\\n - thanosrulers/status\\n - scrapeconfigs\\n - servicemonitors\\n - podmonitors\\n - probes\\n - prometheusrules\\n verbs:\\n - '*'\\n- apiGroups:\\n - apps\\n resources:\\n - statefulsets\\n verbs:\\n - '*'\\n- apiGroups:\\n - \\\"\\\"\\n resources:\\n - configmaps\\n - secrets\\n verbs:\\n - '*'\\n- apiGroups:\\n - \\\"\\\"\\n resources:\\n - pods\\n verbs:\\n - list\\n - delete\\n- apiGroups:\\n - \\\"\\\"\\n resources:\\n - services\\n - services/finalizers\\n verbs:\\n - get\\n - create\\n - update\\n - delete\\n- apiGroups:\\n - \\\"\\\"\\n resources:\\n - nodes\\n verbs:\\n - list\\n - watch\\n- apiGroups:\\n - \\\"\\\"\\n resources:\\n - namespaces\\n verbs:\\n - get\\n - list\\n - watch\\n- apiGroups:\\n - \\\"\\\"\\n resources:\\n - events\\n verbs:\\n - patch\\n - create\\n- apiGroups:\\n - networking.k8s.io\\n resources:\\n - ingresses\\n verbs:\\n - get\\n - list\\n - watch\\n- apiGroups:\\n - storage.k8s.io\\n resources:\\n - storageclasses\\n verbs:\\n - get\\n- apiGroups:\\n - \\\"\\\"\\n resources:\\n - endpoints\\n verbs:\\n - get\\n - create\\n - update\\n - delete\\n```\\n\\nSimilarly to Prometheus, Prometheus Agent will also require permission to scrape targets. Because of this, we will create a new service account for the Agent with the necessary permissions to scrape targets.\\n\\nStart with the ServiceAccount, ClusterRole and ClusterRoleBinding:\\n\\n```yaml mdox-exec=\\\"cat example/rbac/prometheus-agent/prometheus-service-account.yaml\\\"\\napiVersion: v1\\nkind: ServiceAccount\\nmetadata:\\n name: prometheus-agent\\n```\\n\\n```yaml mdox-exec=\\\"cat example/rbac/prometheus-agent/prometheus-cluster-role.yaml\\\"\\napiVersion: rbac.authorization.k8s.io/v1\\nkind: ClusterRole\\nmetadata:\\n name: prometheus-agent\\nrules:\\n- apiGroups: [\\\"\\\"]\\n resources:\\n - services\\n - endpoints\\n - pods\\n verbs: [\\\"get\\\", \\\"list\\\", \\\"watch\\\"]\\n- apiGroups: [\\\"\\\"]\\n resources:\\n - configmaps\\n verbs: [\\\"get\\\"]\\n- apiGroups:\\n - networking.k8s.io\\n resources:\\n - ingresses\\n verbs: [\\\"get\\\", \\\"list\\\", \\\"watch\\\"]\\n- nonResourceURLs: [\\\"/metrics\\\"]\\n verbs: [\\\"get\\\"]\\n```\\n\\n```yaml mdox-exec=\\\"cat example/rbac/prometheus-agent/prometheus-cluster-role-binding.yaml\\\"\\napiVersion: rbac.authorization.k8s.io/v1\\nkind: ClusterRoleBinding\\nmetadata:\\n name: prometheus-agent\\nroleRef:\\n apiGroup: rbac.authorization.k8s.io\\n kind: ClusterRole\\n name: prometheus-agent\\nsubjects:\\n- kind: ServiceAccount\\n name: prometheus-agent\\n namespace: default\\n```\\n\\nLastly, we can deploy the Agent. The `spec` field is very similar to the Prometheus CRD but the features that aren't applicable to the agent mode (like alerting, retention, Thanos, ...) are not available.\\n\\n```yaml mdox-exec=\\\"cat example/rbac/prometheus-agent/prometheus.yaml\\\"\\napiVersion: monitoring.coreos.com/v1alpha1\\nkind: PrometheusAgent\\nmetadata:\\n name: prometheus-agent\\nspec:\\n replicas: 2\\n serviceAccountName: prometheus-agent\\n serviceMonitorSelector:\\n matchLabels:\\n team: frontend\\n```\\n\\nContinue with the [Getting Started page]({{<ref \\\"docs/developer/getting-started.md\\\">}}) to learn how to monitor applications running on Kubernetes.\\n\",\n",
|
|
" \"# Prometheus Agent support\\n\\n## Summary\\n\\nThe Prometheus 2.32.0 release introduces the Prometheus Agent, a mode optimized for remote-write dominant scenarios. This document proposes extending the Prometheus Operator to allow running a Prometheus Agent with different deployment strategies.\\n\\n## Background\\n\\nThe Prometheus Operator in its current state does not allow a simple way of deploying the Prometheus agent. A potential workaround has been described in a [Github comment](https://github.com/prometheus-operator/prometheus-operator/issues/3989#issuecomment-974137486), where the agent can be deployed through the existing Prometheus CRD by explicitly setting command-line arguments specific to the agent mode.\\n\\nAs described in the comment, one significant problem with this approach is that the Prometheus Operator always generates `alerts` and `rules` sections in the Prometheus config file. These sections are not allowed when running the agent so users need to take additional actions to pause reconciliation of the Prometheus CR, tweak the generated secret and then unpause reconciliation in order to resolve the problem. Alternatively, users can apply a strategic merge patch to the prometheus container as described in the kube-prometheus docs: [https://github.com/prometheus-operator/kube-prometheus/blob/main/docs/customizations/prometheus-agent.md](https://github.com/prometheus-operator/kube-prometheus/blob/main/docs/customizations/prometheus-agent.md)\\n\\nWhile this workaround can be used as a stop-gap solution to unblock users in the short term, it has the drawback of needing additional steps which require understanding implementation details of the operator itself. In addition to this, overriding the value of the argument `--config.file` also requires knowledge of Prometheus Operator internals.\\n\\nA lot of the fields supported by the current PrometheusSpec are not applicable to the agent mode. These fields are documented in the PrometheusAgent CRD section.\\n\\nFinally, the Prometheus agent is significantly different from the Prometheus server in the way that it fits in a monitoring stack. Therefore, running it as a StatefulSet might not be the only possible deployment strategy, users might want to run it as a DaemonSet or a Deployment instead.\\n\\n## Proposal\\n\\nThis document proposes introducing a PrometheusAgent CRD to allow users to run Prometheus in agent mode. Having a separate CRD allows the Prometheus and PrometheusAgent CRDs to evolve independently and expose parameters specific to each Prometheus mode.\\n\\nFor example, the PrometheusAgent CRD could have a `strategy` field indicating the deployment strategy for the agent, but no `alerting` field since alerts are not supported in agent mode. Even though there will be an upfront cost for introducing a new CRD, having separate APIs would simplify long-term maintenance by allowing the use of CRD validation mechanisms provided by Kubernetes.\\n\\nIn addition, dedicated APIs with mode-specific fields are self documenting since they remove the need to explicitly document which fields and field values are allowed or required for each individual mode. Users will also be able to get an easier overview of the different parameters they could set for each mode, which leads to a better user experience when using the operator.\\n\\nFinally, the advantage of using a separate CRD is the possibility of using an alpha API version, which would clearly indicate that the CRD is still under development. The Prometheus CRD, on the other hand, has already been declared as v1 and adding experimental fields to it will be challenging from both documentation and implementation aspects.\\n\\n### Prometheus Agent CRD\\n\\nThe PrometheusAgent CRD would be similar to the Prometheus CRD, with the exception of removing fields which are not applicable to the prometheus agent mode.\\n\\nHere is the list of fields we want to exclude:\\n* `retention`\\n* `retentionSize`\\n* `disableCompaction`\\n* `evaluationInterval`\\n* `rules`\\n* `query`\\n* `ruleSelector`\\n* `ruleNamespaceSelector`\\n* `alerting`\\n* `remoteRead`\\n* `additionalAlertRelabelConfigs`\\n* `additionalAlertManagerConfigs`\\n* `thanos`\\n* `prometheusRulesExcludedFromEnforce`\\n* `queryLogFile`\\n* `allowOverlappingBlocks`\\n\\nThe `enabledFeatures` field can be validated for agent-specific features only, which include: `expand-external-labels`, `extra-scrape-metrics` and `new-service-discovery-manager`.\\n\\nFinally, the `remoteWrite` field should be made required only for the agent since it is a mandatory configuration section in agent mode.\\n\\n### Deployment Strategies\\n\\nWhen using Prometheus in server mode, scraped samples are stored in memory and on disk. These samples need to be preserved during disruptions, such as pod replacements or cluster maintenance operations which cause evictions. Because of this, the Prometheus Operator currently deploys Prometheus instances as Kubernetes StatefulSets.\\n\\nOn the other hand, when running Prometheus in agent mode, samples are sent to a remote write target immediately, and are not kept locally for a long time. The only use-case for storing samples locally is to allow retries when remote write targets are not available. This is achieved by keeping scraped samples in a WAL for 2h at most. Samples which have been successfully sent to remote write targets are immediately removed from local storage.\\n\\nSince the Prometheus agent has slightly different storage requirements, this proposal suggests allowing users to choose different deployment strategies.\\n\\n#### Running the agent with cluster-wide scope\\n\\nEven though the Prometheus agent has very little need for storage, there are still scenarios where sample data can be lost if persistent storage is not used. If a remote write target is unavailable and an agent pod is evicted at the same time, the samples collected during the unavailability window of the remote write target will be completely lost.\\n\\nFor this reason, the cluster-wide strategy would be implemented by deploying a StatefulSet, similarly to how `Prometheus` CRs are currently reconciled. This also allows for reusing existing code from the operator and delivering a working solution faster and with fewer changes. Familiarity with how StatefulSets work, together with the possibility to reuse existing code, were the primary reasons for choosing StatefulSets for this strategy over Deployments.\\n\\nThe following table documents the problems that could occur with a Deployment and StatefulSet strategy in different situations.\\n\\n<table>\\n <tr>\\n <td>\\n </td>\\n <td><strong>Pod update</strong>\\n </td>\\n <td><strong>Network outage during pod update</strong>\\n </td>\\n <td><strong>Network outage during node drain</strong>\\n </td>\\n <td><strong>Cloud k8s node rotation</strong>\\n </td>\\n <td><strong>Non-graceful pod deletion</strong>\\n </td>\\n </tr>\\n <tr>\\n <td><strong>Deployment with emptyDir volume</strong>\\n </td>\\n <td>No delay in scrapes if the new pod is created before the old one is terminated\\n </td>\\n <td>Unsent samples will be lost. \\n<p>\\nEmptyDir is tied to a pod <em>and</em> node, and data from the old pod will not be preserved.\\n </td>\\n <td>Unsent samples will be lost. \\n<p>\\nEmptyDir is tied to a pod <em>and</em> node, and data from the old pod will not be preserved.\\n </td>\\n <td>Unsent samples will be lost\\n </td>\\n <td>Unsent samples will be lost. \\n<p>\\nEmptyDir is tied to a pod <em>and</em> node, and data from the old pod will not be preserved.\\n </td>\\n </tr>\\n <tr>\\n <td><strong>Statefulset with a PVC</strong>\\n </td>\\n <td>Potential delay in a subsequent scrape due to recreation of the pod\\n </td>\\n <td>No data loss, the volume will contain all unsent data\\n </td>\\n <td>No data loss, the volume will contain all unsent data\\n </td>\\n <td>No data loss if a new pod scheduled to the same AZ node. May be stuck in pending state otherwise\\n </td>\\n <td>No data loss, the volume will contain all unsent data\\n </td>\\n </tr>\\n <tr>\\n <td><strong>Deployment or STS with replicas</strong>\\n </td>\\n <td>No delay, mitigated by replicas\\n </td>\\n <td>Unsent data will be lost if last replica terminated before network outage resolves\\n </td>\\n <td>No data loss, as other replicas are running on other nodes\\n </td>\\n <td>No data loss, as other replicas running on other nodes\\n </td>\\n <td>No data loss as other replicas untouched\\n </td>\\n </tr>\\n</table>\\n\\n#### Running the agent with node-specific scope\\n\\nThis strategy has a built-in auto-scaling mechanism since each agent will scrape only a subset of the targets. As the cluster grows and more nodes are added to it, new agent instances will automatically be scheduled to scrape pods on those nodes. Even though the load distribution will not be perfect (targets on certain nodes might produce far more metrics than targets on other nodes), it is a simple way of adding some sort of load management.\\n\\nAnother advantage is that persistent storage can now be handled by mounting a host volume, a strategy commonly used by log collectors. The need for persistent storage is described in the StatefulSet strategy section.\\n\\nThe Grafana Agent config exposes a `host_filter` boolean flag which, when enabled, instructs the agent to only filter targets from the same node, in addition to the scrape config already provided. With this option, the same config can be used for agents running on multiple nodes, and the agents will automatically scrape targets from their own nodes. Such a config option is not yet available in Prometheus. An issue has already been raised [[3]](https://github.com/prometheus/prometheus/issues/9637) and there is an open PR for addressing it [[4]](https://github.com/prometheus/prometheus/pull/10004).\\n\\nUntil the upstream work has been completed, it could be possible to implement this strategy with a few tweaks:\\n* the operator could use the [downward API](https://kubernetes.io/docs/tasks/inject-data-application/downward-api-volume-expose-pod-information/#capabilities-of-the-downward-api) to inject the node name in the pods.\\n* the operator's config reloader already supports expansion of environment variables.\\n\\nWith this setup, the unexpanded Prometheus configuration would look as follows\\n\\n```yaml\\nrelabel_configs:\\n- source_labels: [__meta_kubernetes_pod_node_name]\\n action: keep\\n regex: $NODE_NAME\\n\\nin the pod definition:\\nspec:\\n- container: config-reloader\\n env:\\n- name: NODE_NAME\\n valueFrom:\\n fieldRef:\\n fieldPath: spec.nodeName\\n```\\n\\n## Additional implementation details\\n\\nThere has been a suggestion in [a Github comment](https://github.com/prometheus-operator/prometheus-operator/issues/3989#issuecomment-821249404) to introduce a ScrapeConfig CRD in parallel to adding the PrometheusAgent CRD, and \\u201ctranslate\\u201d PrometheusAgent CRs to ScrapeConfig CRs. The main challenge with this approach is that it significantly increases the scope of the work that needs to be done to support deploying Prometheus agents.\\n\\nA leaner alternative would be to focus on implementing the PrometheusAgent CRD by reusing code from the existing Prometheus controller. The ScrapeConfig can then be introduced separately, and the PrometheusAgent can be the first CRD which gets migrated to it.\\n\\n### Implementation steps\\n\\nThe first step in the implementation process would include creating the PrometheusAgent CRD and deploying the agent as a StatefulSet, similar to how the Prometheus CRD is currently reconciled. This will allow for reusing a lot of the existing codebase from the Prometheus controller and the new CRD can be released in a timely manner.\\n\\nSubsequent steps would include iterating on users' feedback and either implementing different deployment strategies, or refining the existing one.\\n\\n## References\\n* [1] [https://github.com/grafana/agent/blob/5bf8cf452fa76c75387e30b6373630923679221c/production/kubernetes/agent-bare.yaml#L43](https://github.com/grafana/agent/blob/5bf8cf452fa76c75387e30b6373630923679221c/production/kubernetes/agent-bare.yaml#L43)\\n* [2] [https://github.com/open-telemetry/opentelemetry-operator#deployment-modes](https://github.com/open-telemetry/opentelemetry-operator#deployment-modes)\\n* [3] [https://github.com/prometheus/prometheus/issues/9637](https://github.com/prometheus/prometheus/issues/9637)\\n* [4] [https://github.com/prometheus/prometheus/pull/10004](https://github.com/prometheus/prometheus/pull/10004)\\n\",\n",
|
|
" \"# Additional Scrape Configuration\\n\\nAdditionalScrapeConfigs allows specifying a key of a Secret containing\\nadditional Prometheus scrape configurations. Scrape configurations specified\\nare appended to the configurations generated by the Prometheus Operator.\\n\\nJob configurations specified must have the form as specified in the official\\n[Prometheus documentation](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#scrape_config).\\nAs scrape configs are appended, the user is responsible to make sure it is\\nvalid. *Note* that using this feature may expose the possibility to break\\nupgrades of Prometheus.\\n\\nIt is advised to review Prometheus release notes to ensure that no incompatible\\nscrape configs are going to break Prometheus after the upgrade.\\n\\n## Creating an additional configuration\\n\\nFirst, you will need to create the additional configuration.\\nBelow we are making a simple \\\"prometheus\\\" config. Name this\\n`prometheus-additional.yaml` or something similar.\\n\\n```yaml\\n- job_name: \\\"prometheus\\\"\\n static_configs:\\n - targets: [\\\"localhost:9090\\\"]\\n```\\n\\nThen you will need to make a secret out of this configuration.\\n\\n```sh\\nkubectl create secret generic additional-scrape-configs --from-file=prometheus-additional.yaml --dry-run=client -oyaml > additional-scrape-configs.yaml\\n```\\n\\nNext, apply the generated kubernetes manifest\\n\\n```\\nkubectl apply -f additional-scrape-configs.yaml -n monitoring\\n```\\n\\nFinally, reference this additional configuration in your `prometheus.yaml` CRD.\\n\\n```yaml\\napiVersion: monitoring.coreos.com/v1\\nkind: Prometheus\\nmetadata:\\n name: prometheus\\n labels:\\n prometheus: prometheus\\nspec:\\n replicas: 2\\n serviceAccountName: prometheus\\n serviceMonitorSelector:\\n matchLabels:\\n team: frontend\\n additionalScrapeConfigs:\\n name: additional-scrape-configs\\n key: prometheus-additional.yaml\\n```\\n\\nNOTE: Use only one secret for ALL additional scrape configurations.\\n\\n## Additional References\\n\\n* [Prometheus Spec](api.md#monitoring.coreos.com/v1.PrometheusSpec)\\n* [Additional Scrape Configs](../example/additional-scrape-configs)\\n\"\n",
|
|
" ]\n",
|
|
" ],\n",
|
|
" \"uris\": null,\n",
|
|
" \"data\": null,\n",
|
|
" \"metadatas\": [\n",
|
|
" [\n",
|
|
" null,\n",
|
|
" null,\n",
|
|
" null\n",
|
|
" ]\n",
|
|
" ],\n",
|
|
" \"distances\": [\n",
|
|
" [\n",
|
|
" 0.835404098033905,\n",
|
|
" 0.8792126774787903,\n",
|
|
" 0.8795778155326843\n",
|
|
" ]\n",
|
|
" ],\n",
|
|
" \"included\": [\n",
|
|
" \"distances\",\n",
|
|
" \"documents\",\n",
|
|
" \"metadatas\"\n",
|
|
" ]\n",
|
|
"}\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"import json # we will use this to pretty-print the result\n",
|
|
"\n",
|
|
"result = collection.query(\n",
|
|
" query_texts=[\"This is a document about prometheus.\"],\n",
|
|
" n_results=3, # how many results to return (10 by default)\n",
|
|
")\n",
|
|
"print(json.dumps(result, indent=2))"
|
|
]
|
|
}
|
|
],
|
|
"metadata": {
|
|
"kernelspec": {
|
|
"display_name": "env",
|
|
"language": "python",
|
|
"name": "python3"
|
|
},
|
|
"language_info": {
|
|
"codemirror_mode": {
|
|
"name": "ipython",
|
|
"version": 3
|
|
},
|
|
"file_extension": ".py",
|
|
"mimetype": "text/x-python",
|
|
"name": "python",
|
|
"nbconvert_exporter": "python",
|
|
"pygments_lexer": "ipython3",
|
|
"version": "3.13.1"
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 2
|
|
}
|