-
Notifications
You must be signed in to change notification settings - Fork 10
Create and Update the validation service #97
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Changes from 19 commits
5fc76ea
481b382
4e5dfc3
95b565b
9cc270d
e0551ca
8942860
fec88d0
ee5749e
4ba7cb3
ce3dde2
be32c2f
87ec990
f3e0665
a6115c2
42d298b
f41e172
aef7f85
90fc1bd
cfb3e40
8d673d6
850a048
1f43986
1df27bf
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,22 @@ | ||
| # Validation service | ||
|
|
||
| The validation service for Canis Major performs security checks to verify transaction integrity between the context broker and Canis Major blockchain transactions. | ||
|
|
||
|  | ||
|
|
||
|
|
||
| ## Service Setup | ||
| The validation service implements a three-step verification protocol. | ||
|
|
||
| ### Initial Validation Request | ||
| The client initiates the verification process by submitting a POST request to the Canis Major Validation service. | ||
|
|
||
| ### Dual-Source Data Retrieval | ||
| The Canis Major Validator executes two GET requests, based on the `entityId`, to: | ||
| - Canis Major | ||
| - The Context Broker | ||
|
|
||
| Both requests retrieve the entity information using the specified entity identifier as the query parameter. | ||
|
|
||
| ### Data Integrity Analysis | ||
| The validator performs a systematic comparison of the entity representations obtained from both sources. | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,65 @@ | ||
| <?xml version="1.0" encoding="UTF-8"?> | ||
| <project xmlns="http://maven.apache.org/POM/4.0.0" | ||
| xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" | ||
| xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> | ||
| <modelVersion>4.0.0</modelVersion> | ||
|
|
||
| <groupId>validator</groupId> | ||
| <artifactId>validation-service</artifactId> | ||
| <version>1.0-SNAPSHOT</version> | ||
| <packaging>jar</packaging> | ||
|
|
||
| <properties> | ||
| <java.version>17</java.version> <!-- Specify your Java version --> | ||
| <jakarta.version>2.0.0</jakarta.version> <!-- Adjust as necessary --> | ||
| <jackson.version>2.13.0</jackson.version> <!-- Adjust as necessary --> | ||
| <slf4j.version>1.7.32</slf4j.version> <!-- Adjust as necessary --> | ||
| </properties> | ||
|
|
||
| <dependencies> | ||
| <dependency> | ||
| <groupId>jakarta.inject</groupId> | ||
| <artifactId>jakarta.inject-api</artifactId> | ||
| <version>${jakarta.version}</version> | ||
| </dependency> | ||
| <dependency> | ||
| <groupId>com.fasterxml.jackson.core</groupId> | ||
| <artifactId>jackson-databind</artifactId> | ||
| <version>${jackson.version}</version> | ||
| </dependency> | ||
| <dependency> | ||
| <groupId>org.slf4j</groupId> | ||
| <artifactId>slf4j-api</artifactId> | ||
| <version>${slf4j.version}</version> | ||
| </dependency> | ||
| <dependency> | ||
| <groupId>org.slf4j</groupId> | ||
| <artifactId>slf4j-simple</artifactId> | ||
| <version>${slf4j.version}</version> | ||
| </dependency> | ||
| <dependency> | ||
| <groupId>io.micronaut</groupId> | ||
| <artifactId>micronaut-http-client</artifactId> | ||
| <version>3.5.0</version> <!-- Adjust as necessary --> | ||
| </dependency> | ||
| <dependency> | ||
| <groupId>io.micronaut</groupId> | ||
| <artifactId>micronaut-runtime</artifactId> | ||
| <version>3.5.0</version> <!-- Adjust as necessary --> | ||
| </dependency> | ||
| </dependencies> | ||
|
|
||
| <build> | ||
| <plugins> | ||
| <plugin> | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. What is the further plan? At the moment no container will be build. Will there be another PR adding functionality? |
||
| <groupId>org.apache.maven.plugins</groupId> | ||
| <artifactId>maven-compiler-plugin</artifactId> | ||
| <version>3.8.1</version> | ||
| <configuration> | ||
| <source>${java.version}</source> | ||
| <target>${java.version}</target> | ||
| </configuration> | ||
| </plugin> | ||
| </plugins> | ||
| </build> | ||
| </project> | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,54 @@ | ||
| package validator.client; | ||
|
|
||
| // Import statements: | ||
| import jakarta.inject.Singleton; // For dependency injection, marks class as singleton | ||
| import java.net.URI; // For handling URIs | ||
| import java.net.http.HttpClient; // Core class for making HTTP requests | ||
| import java.net.http.HttpRequest; // For building HTTP requests | ||
| import java.net.http.HttpResponse; // For handling HTTP responses | ||
|
|
||
|
|
||
| // Marks this class as a singleton - only one instance will exist in the application | ||
| @Singleton | ||
| public class ValidationHttpClient { | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It would be nicer if the methods of this class would return meaningful objects rather than unchecked strings |
||
| // Creates a single HttpClient instance to be reused for all requests | ||
| private final HttpClient client = HttpClient.newHttpClient(); | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Might be to much for the current level but its possible to use a generated client for the ngsi-ld standard. Either you generate it yourself in this project -> https://github.com/wistefan/ngsi-ld-java-mapping/blob/main/pom.xml#L400-L442 or you could use an even more abstract approach using the lib to access ngsi-ld context broker via a convenient repository: https://github.com/wistefan/ngsi-ld-java-mapping/tree/main |
||
|
|
||
| // Method to fetch data from Orion Context Broker | ||
| public String fetchOrionData(String entityId) throws Exception { | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Canis Major does not request Orion-ld but a NGSI_LD compatible broker right? Then the name would be misleading
Collaborator
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yes, I agree. CanisMajor works with NGSI-LD and even NGSI-V2 as well.
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @asmataamallah25 It does not work with v2. |
||
| // Builds an HTTP request: | ||
| HttpRequest request = HttpRequest.newBuilder() | ||
| // Sets the URI for the request, querying for DLTtxReceipt entities with matching refEntity | ||
| .uri(new URI("http://localhost:1026/ngsi-ld/v1/entities/?type=DLTtxReceipt&q=refEntity==" + entityId)) | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The broker location should be configurable |
||
| // Adds Link header for JSON-LD context | ||
| .header("Link", "<https://raw.githubusercontent.com/smart-data-models/dataModel.DistributedLedgerTech/master/context.jsonld>; rel=\"http://www.w3.org/ns/json-ld#context\"; type=\"application/ld+json\"") | ||
| // Sets the NGSILD-Tenant header to "orion" | ||
| .header("NGSILD-Tenant", "orion") | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Tenant should be configurable too |
||
| // Added Accept header for JSON format | ||
| .header("Accept", "application/json") | ||
| // Specifies this is a GET request | ||
| .GET() | ||
| // Finalizes the request building | ||
| .build(); | ||
|
|
||
| // Sends the request and returns the response body as a String | ||
| return client.send(request, HttpResponse.BodyHandlers.ofString()).body(); | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. For future maintainability the use of async mechanisms would be nice, or else we can run into annoying errors when running it |
||
| } | ||
|
|
||
| // Method to fetch data from a blockchain service | ||
| public String fetchBlockchainData(String entityId) throws Exception { | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is it really a blockchain service (somehow generic) or is it CanisMajor?
Collaborator
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This service is a related to Canis Major. |
||
| // Builds an HTTP request: | ||
| HttpRequest request = HttpRequest.newBuilder() | ||
| // Sets the URI for the blockchain service endpoint | ||
| .uri(new URI("http://localhost:4000/entity/" + entityId)) | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Configurable please 😄 |
||
| // Sets Accept header to receive JSONresponse | ||
| .header("Accept", "application/json") | ||
| // Specifies this is a GET request | ||
| .GET() | ||
| // Finalizes the request building | ||
| .build(); | ||
|
|
||
| // Sends the request and returns the response body as a String | ||
| return client.send(request, HttpResponse.BodyHandlers.ofString()).body(); | ||
| } | ||
| } | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,36 @@ | ||
| package validator.controller; | ||
|
|
||
| import io.micronaut.http.annotation.*; | ||
| import io.micronaut.http.HttpResponse; | ||
| import validator.service.ValidationService; | ||
| import validator.model.ValidationResult; | ||
|
|
||
| @Controller("/service/v1/validation") | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Would be nice to have an openapi file and generate the api from it (like here https://github.com/FIWARE/CanisMajor/blob/master/src/main/java/org/fiware/canismajor/rest/EntitiesController.java#L21) |
||
| public class ValidationController { | ||
| private final ValidationService validationService; | ||
|
|
||
| public ValidationController(ValidationService validationService) { | ||
| this.validationService = validationService; | ||
| } | ||
|
|
||
| @Post("/compare") | ||
| public HttpResponse<ValidationResult> compareResponses(@Body CompareRequest request) { | ||
| // Call the validation service to perform the validation | ||
| ValidationResult result = validationService.validateEntity(request.getEntityId()); | ||
| return HttpResponse.ok(result); | ||
| } | ||
|
|
||
| // Inner class to represent the request body | ||
| public static class CompareRequest { | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Would be generated too |
||
| private String entityId; | ||
|
|
||
| // Getter and Setter | ||
| public String getEntityId() { | ||
| return entityId; | ||
| } | ||
|
|
||
| public void setEntityId(String entityId) { | ||
| this.entityId = entityId; | ||
| } | ||
| } | ||
| } | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,30 @@ | ||
| package validator.model; | ||
|
|
||
| import java.util.List; | ||
|
|
||
| //import java.util.List; | ||
|
|
||
| /*public record ValidationResult(boolean isValid, List<String> errors) { | ||
| public String getErrorMessage() { | ||
| return String.join(", ", errors); | ||
| } | ||
| } */ | ||
|
|
||
| public class ValidationResult { | ||
| private final boolean valid; | ||
| private final List<String> messages; | ||
|
|
||
|
|
||
| public ValidationResult(boolean valid, List<String> messages){ | ||
| this.valid = valid; | ||
| this.messages = messages; | ||
| } | ||
|
|
||
| public boolean isValid() { //getter | ||
| return valid; | ||
| } | ||
|
|
||
| public List<String> getMessages() { | ||
| return messages; | ||
| } | ||
| } |
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -0,0 +1,93 @@ | ||||||
| /* | ||||||
| This implementation of the validation service: | ||||||
| - Compares all relevant fields between Orion and blockchain responses | ||||||
| - Provides detailed validation errors for each mismatched field | ||||||
| - Handles JSON parsing and null checks | ||||||
| - Uses logging for error tracking | ||||||
| - Returns structured validation results | ||||||
| */ | ||||||
|
|
||||||
| package validator.service; | ||||||
|
|
||||||
| import com.fasterxml.jackson.databind.JsonNode; | ||||||
| import com.fasterxml.jackson.databind.ObjectMapper; | ||||||
| import com.fasterxml.jackson.core.JsonProcessingException; | ||||||
| import validator.model.ValidationResult; | ||||||
| import jakarta.inject.Singleton; | ||||||
| import validator.client.ValidationHttpClient; | ||||||
| import org.slf4j.Logger; | ||||||
| import org.slf4j.LoggerFactory; | ||||||
| import java.util.ArrayList; | ||||||
| import java.util.List; | ||||||
|
|
||||||
| @Singleton | ||||||
| public class ValidationService { | ||||||
| private static final Logger logger = LoggerFactory.getLogger(ValidationService.class); | ||||||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Just for fun you should check out Lombok. Its a very convenient tool to reduce boilerplate |
||||||
| private final ValidationHttpClient validationHttpClient; | ||||||
| private final ObjectMapper mapper = new ObjectMapper(); | ||||||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Micronaut in its basic/default config lets you inject an ObjectMapper afaik |
||||||
|
|
||||||
| public ValidationService(ValidationHttpClient validationHttpClient) { | ||||||
| this.validationHttpClient = validationHttpClient; | ||||||
| } | ||||||
|
|
||||||
| public ValidationResult validateEntity(String entityId) { | ||||||
| try { | ||||||
| String orionData = validationHttpClient.fetchOrionData(entityId); | ||||||
| String blockchainData = validationHttpClient.fetchBlockchainData(entityId); | ||||||
|
|
||||||
| if (orionData == null || orionData.isEmpty()) { | ||||||
| logger.error("No data found from Orion for entityId: {}", entityId); | ||||||
| return new ValidationResult(false, List.of("No data found from Orion")); | ||||||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
| } | ||||||
|
|
||||||
| if (blockchainData == null || blockchainData.isEmpty()) { | ||||||
| logger.error("No data found from Canis Major for entityId: {}", entityId); | ||||||
| return new ValidationResult(false, List.of("No data found from Canis Major")); | ||||||
| } | ||||||
|
|
||||||
| List<String> validationErrors = compareData(orionData, blockchainData); | ||||||
| return new ValidationResult(validationErrors.isEmpty(), validationErrors); | ||||||
| } catch (Exception e) { | ||||||
| logger.error("Error validating entity: {}", entityId, e); | ||||||
| return new ValidationResult(false, List.of("Validation error: " + e.getMessage())); | ||||||
| } | ||||||
| } | ||||||
|
|
||||||
| private List<String> compareData(String orionData, String blockchainData) { | ||||||
| List<String> errors = new ArrayList<>(); | ||||||
| try { | ||||||
| JsonNode orionNode = mapper.readTree(orionData); | ||||||
| JsonNode blockchainNode = mapper.readTree(blockchainData); | ||||||
|
|
||||||
| JsonNode txReceipts = orionNode.get("TxReceipts").get("value"); | ||||||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Possible NPE, better have the clients return checked objects |
||||||
|
|
||||||
| validateField(txReceipts, blockchainNode, "status", "Status", errors); | ||||||
| validateField(txReceipts, blockchainNode, "blockNumber", "Block Number", errors); | ||||||
| validateField(txReceipts, blockchainNode, "transactionHash", "Transaction Hash", errors); | ||||||
| validateField(txReceipts, blockchainNode, "blockHash", "Block Hash", errors); | ||||||
| validateField(txReceipts, blockchainNode, "from", "From Address", errors); | ||||||
| validateField(txReceipts, blockchainNode, "to", "To Address", errors); | ||||||
|
|
||||||
| return errors; | ||||||
| } catch (JsonProcessingException e) { | ||||||
| logger.error("Error parsing JSON data", e); | ||||||
| errors.add("Error parsing JSON data: " + e.getMessage()); | ||||||
| return errors; | ||||||
| } | ||||||
| } | ||||||
|
|
||||||
| private void validateField(JsonNode orion, JsonNode blockchain, String fieldName, String displayName, List<String> errors) { | ||||||
| JsonNode orionValue = orion.get(fieldName); | ||||||
| JsonNode blockchainValue = blockchain.get(fieldName); | ||||||
|
|
||||||
| if (orionValue == null || blockchainValue == null) { | ||||||
| errors.add(displayName + " field missing in one or both responses"); | ||||||
| return; | ||||||
| } | ||||||
|
|
||||||
| if (!orionValue.equals(blockchainValue)) { | ||||||
| errors.add(displayName + " mismatch: Orion=" + orionValue + ", Blockchain=" + blockchainValue); | ||||||
| } | ||||||
| } | ||||||
| } | ||||||
|
|
||||||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,12 @@ | ||
| #Stop and remove all Docker containers: | ||
|
flopezag marked this conversation as resolved.
Outdated
|
||
| docker stop $(sudo docker ps -aq) | ||
| docker rm $(sudo docker ps -aq) | ||
| docker container prune | ||
|
|
||
| #Remove all created volumes: | ||
| docker volume rm $(sudo docker volume ls -q) | ||
| docker volume prune | ||
|
|
||
| #Remove all Docker custom networks: | ||
| docker network prune -f | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why didn't you put this in a separate repo? Currently the service is placed in the repo but not included in the build system ( neither as submodule nor in the github flows).