codexworker.com https://www.codexworker.com personal homepage of backend developer writing random stuff Wed, 30 Apr 2025 04:59:25 +0000 Today’s Development Highlights: UI/UX, Security & Code Improvements https://www.codexworker.com/post/133/a What developments have taken place today? Today’s commits focused on enhancing user interfaces, improving backend functionality, and refining existing components. The first set of changes streamlined the homepage layout by introducing a component-based architecture using `render_component`, resulting in cleaner code and easier maintainability. This involved replacing static HTML elements with reusable components for “hero” and “features,” alongside CSS imports to style these new sections and updating the overall page structure. The second commit addressed password functionality, introducing a robust password validation mechanism within the remote daemon client and enhancing user profile editing capabilities. Specifically, it implemented a `NewPasswordValidator` with minimum length requirements, displayed timezone and locale information on the profile edit page, and added a "Save" button for easy updates. These changes ensure secure password management and improved user control over their settings. A third commit focused on refining email communication related to password recovery and registration. It replaced text-based links with HTML buttons in password recovery emails, included personalized content through variables like `userNick`, and added a fallback option for users unable to click the link. Furthermore, it implemented status message handling and layout improvements for these forms, providing immediate feedback to users during the process. Finally, several smaller changes were made including fixing a bug in test cases related to website URL retrieval, updating the application name throughout the HTML code, modifying import statements within stylesheets, improving the user detail page with a breadcrumb component and email verification status display, and updating password change page templates. These combined updates contribute to a more robust, user-friendly, and maintainable codebase. In essence, these changes represent significant improvements in UI/UX, backend security, and code organization, leading to a better overall application experience. Mon, 28 Apr 2025 23:59:59 +0000 Enhanced Date Transparency: New Tooltips Reveal Creation Dates & Times https://www.codexworker.com/post/132/a What developments have taken place today? A tooltip has been implemented to display detailed creation dates and times for each date element within the interface. Hovering over a specific date now reveals the precise timestamp of its corresponding entry. This enhancement provides users with immediate access to granular information, improving clarity and usability. This change improves user experience by providing contextual details on demand, reducing the need for users to manually search for or calculate dates. The tooltip functionality streamlines data interpretation and offers a more intuitive way to understand the timeline associated with each entry. In essence, these updates significantly enhance transparency and accessibility of information presented within the system, leading to a more informed and efficient user experience. Mon, 28 Apr 2025 23:59:59 +0000 Automatic information https://www.codexworker.com/post/130/a Hello! Code change information from bot here! During week 2025-04-21 - 2025-04-27 i made 214 commits across 5 of my projects. In total i created 26096 new lines and removed 20970 lines. Mon, 28 Apr 2025 00:02:39 +0000 Website Updates: Improvements & New Features https://www.codexworker.com/post/131/a What developments have taken place today? Several improvements and refinements have been implemented across various parts of the website’s user interface and backend data handling. These updates focus on enhancing presentation, improving search functionality, optimizing database structures, and ensuring consistent styling. Specifically, date formatting has been adjusted for visual clarity, the maximum number of posts returned in searches has been significantly increased, and a modern grid layout has been introduced for displaying project statistics, incorporating responsive design principles. A key enhancement involves the addition of tag support to post listings, achieved through new template functions and database schema modifications. This allows for associating tags with posts, presenting them as clickable links alongside post information. Furthermore, significant changes have been made to the display of weekly project statistics, prioritizing current relevance and utilizing a grid layout that adapts to different screen sizes. The breadcrumb navigation has also received attention, with adjustments to font size, color, background styling, and margin to improve visual hierarchy and prominence. Database modifications include adding a “tags” column to the “post” table, alongside an accompanying migration script for rollback purposes. This addition is coupled with the removal of previously used "slug" and "tags" columns. Additionally, several stylistic updates have been implemented across various elements, including links within breadcrumbs and text sizes/colors to ensure visual consistency. Finally, a substantial increase in the number of posts returned by search queries has been made, intending to improve user experience through expanded results. These changes collectively contribute to a more organized, visually appealing, and informative website experience for users, while also laying groundwork for future enhancements like tag-based filtering and improved data management within the database. Sun, 27 Apr 2025 23:59:59 +0000 Today’s Website Developments: Enhancements & New Features https://www.codexworker.com/post/129/a What developments have taken place today? This commit modified data insertion into a table by adding a `slug` field to the `title` column and updating existing post slugs ("123-slug" and “22-slug”). All other fields remain unchanged, improving data organization and URL structure. A website navigation and breadcrumb overhaul was implemented, enhancing visual consistency and clarity. Specifically, visited links in the main navigation and breadcrumbs now appear blue, and the breadcrumb display shows "Home" alongside the website name, offering improved user guidance. Additionally, a recent development posts section was added to facilitate content discovery. The HTML template was updated to introduce `<section>` elements styled as “article” for each blog post, incorporating styling through the new `articles.scss` stylesheet and adjusting spacing, container width, and table styles for a more refined layout. A new website detail page was introduced, showcasing project information alongside related posts using templates like `render_posts.go.tmpl`. This provides a consolidated view of key project data, including description, recent activity, and associated content. The weekly stats are fetched from Gitlab via a new repository, offering real-time development insights. The contact form was removed, streamlining the website’s interface. Template updates (`header.go.tmpl` and `footer.go.tmpl`) were applied alongside basic styling for container, section, and table elements, improving overall visual presentation. The Gitlab project stats repo was updated with new fields for weekly change data, and a placeholder was added for a newsletter signup form. A date formatting function (`humanReadableTimeDifference`) was introduced in `date_format.ts` to provide human-readable date representations, enhancing the clarity of post creation dates on the website. The "About Me" section was redesigned with a photo and updated description emphasizing relevant skills. A table displaying project changes, including commit counts, lines added/removed, and repository size, was implemented for enhanced tracking. Project information visibility was controlled by user role, providing improved access management. Finally, analytics tables were created to track user sessions. Sensitive database credentials and remote host information have been removed from the configuration, enhancing security by preventing accidental exposure. A check has also been added to gracefully terminate the application if a database connection error occurs. Website styling and layout enhancements were implemented, including a new breadcrumb navigation system with links to "Home" and project pages. Page titles are now styled for increased prominence, and navigation link colors have been standardized to white with a hover effect. Visited link styles have been corrected to match the design aesthetic. The `ORDER BY` clause in the Gitlab project stats query was adjusted for proper sorting. The header has been updated to include an anchor tag linking to the homepage and wrapping the title in an `<h1>` tag. A set of CSS variables were defined at the `:root` level, establishing a consistent color palette across the application. These variables control primary, dark, light background, and accent colors for greater design flexibility. Additionally, spacing and padding were adjusted throughout the design to ensure visual harmony. The HTML background color was updated to a lighter off-white shade, and the margin below sections was reduced for improved readability. A slug generation feature has been implemented for post creation, ensuring that URLs are consistently formatted and user-friendly. The `PostRepository` now utilizes the `github.com/gosimple/slug` library to generate slugs from post titles. A database migration was applied to add a dedicated `slug` column to the `post` table and populate it with generated slugs for existing posts. Analytics data collection has been introduced through new tables (`analytics`, `analytics_url`). These tables store information about user sessions, including cookie IDs, locations, durations, IP addresses, and referrer URLs. This enhanced tracking provides valuable insights into website usage patterns. Finally, a message is now displayed if a website does not have a publicly accessible URL, informing users of the situation instead of omitting the link. Summary: These changes focused on improving data organization, enhancing user experience through navigation and layout refinements, securing sensitive information, adding new features like slug generation and analytics tracking, and streamlining the overall presentation of project details and recent activity on the website. Sat, 26 Apr 2025 23:59:59 +0000 Today's Website Updates: Key Developments & Improvements https://www.codexworker.com/post/104/a What developments have taken place today? This commit modifies data insertion into a table by adding a `slug` field to the `title` column and updating existing post values. The first post's slug is “123-slug” and the second’s is “22-slug”. This improves data organization and URL generation for posts. A website navigation and breadcrumb update enhances visual consistency. Visited links in navigation and breadcrumbs are now blue, and both "Home" and the website name appear in the breadcrumb display. Recent development posts are also added with links to individual post pages, providing users with easy access to updates. The HTML template has been updated to include `<section>` elements for each blog post, styled as “article,” along with associated CSS styling via `articles.scss`. Specific changes include a new `article-meta` div for post creation dates, adjusted spacing and container dimensions, a white table background, and modified markdown output to return raw HTML instead of replacing newline characters. A new project detail page has been introduced, showcasing project information and related posts. It includes a header section with basic details, a project overview, a list of recent posts, weekly Gitlab statistics (with updated repository fields), and the removal of the contact form. Template updates include `header.go.tmpl` and `footer.go.tmpl`, along with basic styling for container, sections, and tables. Several security improvements have been implemented: database credentials and remote host information have been removed from the configuration, and a check is now in place to halt program execution if a database connection error occurs. Styling and layout changes improve website navigation and page structure. Breadcrumb navigation has been added with links to "Home" and the "Projects" page. Page title styling has also been implemented, alongside updated navigation link colors (white on hover turning to `#0d6efd`). Visited link styles are now white for consistency, and the `ORDER BY` clause in the Gitlab stats query has been changed from descending commit count to descending week year. A new display feature is introduced with `date_format.ts`, allowing dates to be formatted as human-readable strings. The website details page now displays project information like name and description. Post lists are rendered using `render_posts.go.tmpl`. Breadcrumbs now correctly include both the website name and post title. Card styling has been added, including subtle shadows, hover effects, and rounded corners. Unnecessary sections have been commented out. The post list now shows dates and titles in a more compact format with data attributes. A color palette is established using CSS variables at the `:root` level to define primary, dark, light background colors and accent colors. Minor adjustments include adding `.env` and `tables.txt` files, removing an existing `tables.txt` file, adjusting spacing and padding, modifying HTML element backgrounds, and deferring the loading of a JavaScript file referencing `/js/date_format.js`. Finally, a conditional display is implemented for project names and URLs based on user role, ensuring non-administrators do not see sensitive information. In summary, these changes represent significant improvements to the website’s usability, organization, security, and data presentation, enhancing both its visual appeal and functionality. Sat, 26 Apr 2025 23:59:59 +0000 Today’s Website Updates: Data, Design & Analytics Improvements https://www.codexworker.com/post/95/a What developments have taken place today? This commit modified data insertion into a table by adding a `slug` field to the `title` column for both posts, assigning "123-slug" and “22-slug” respectively. All other fields remain unchanged. This improves data organization and URL friendliness. The website’s navigation and breadcrumb structure were updated with blue styling for visited links in the main navigation and breadcrumbs, along with a refactored breadcrumb display showing both "Home" and the website name. A section displaying recent development posts was also added with links to individual post pages. These changes improve visual consistency and user experience by guiding users through the site hierarchy. The HTML template was updated to include a new `<section>` element for each blog post, styled as an “article,” incorporating styling for the article container and content using the `articles.scss` stylesheet. This provides a consistent structure for displaying posts and improves visual presentation. A new website detail page was introduced showcasing project information and related posts. It features a header section with basic details, a project overview, a list of recent posts associated with the project, weekly Gitlab stats presented in a table, and (previously) a contact form that has been removed. Template updates (`header.go.tmpl` and `footer.go.tmpl`) were also implemented, along with styling and updated gitlab project stats repo fields. This consolidates key information into a single location for easy access. A function to format dates into human-readable strings was added in `date_format.ts`, facilitating clearer date display on the website. The template now renders post lists using `render_posts.go.tmpl` rather than directly within the main layout, leading to cleaner code and easier maintenance. Breadcrumbs were updated to include both "Home" and the current post title. Card styling was added with subtle shadows and hover effects. Unnecessary sections were commented out. The post list now displays both date and title in a compact format with data attributes. A CSS variables file (`:root`) was created, defining color palettes for primary, dark, light backgrounds, and accent colors to ensure consistent design across the application. Several files were added or updated including `docker/.env`, `shell/tables.txt` (replaced by its content), and a database migration that adds a slug column to the `post` table. Additionally, several external dependencies have been added to the project's Go module file (`go.mod`). These changes enhance maintainability and provide a foundation for future design refinements. The page displaying project changes was enhanced with breadcrumbs, an “About Me” section redesign, and a table for presenting project data including commit counts, lines added/removed, and repository size. Sorting by change volume and conditional visibility based on user role were implemented. This provides more comprehensive project tracking information to the user. A new analytics system was introduced with tables `analytics` and `analytics_url`, capturing session data like cookie IDs, locations, durations, and IP addresses. The implementation of a slug generation feature in the post creation process ensures consistent and shareable URLs. These features enable improved website analysis and tracking capabilities. Finally, minor adjustments were made to styling, including spacing and padding refinements, a lighter background color, and a new button style (`.btn-link-alt`) for alternative link appearances. In summary, today's changes focused on improving the organization, presentation, and functionality of the website through data structure enhancements, visual updates, analytics implementation, and streamlined code management, all contributing to a more user-friendly and informative experience. Sat, 26 Apr 2025 23:59:59 +0000 Remote Daemon Updates: Key Improvements Today https://www.codexworker.com/post/128/a What developments have taken place today? Today’s updates primarily focus on enhancing user experience and improving backend infrastructure for email processing and Lighthouse monitoring within the remote daemon system. Key changes include: implementing a password recovery flow with associated templates, updating Docker container configurations for email and lighthouse consumers, refining the registration page layout, introducing new RabbitMQ consumer services, and updating various dependencies. The core development effort has shifted towards providing robust mechanisms for user authentication and data monitoring. The most significant improvements involve the addition of a functional password recovery system. This encompasses creating a dedicated controller (`PasswordRecoveryController`), defining a request structure (`PasswordRecoveryReq`), generating email templates, and implementing the logic to send reset links via email. Furthermore, the registration page has been revamped with better visual presentation and improved error handling through status messages. On the infrastructure side, two new Docker containers – `remotedaemon_com_consumer_email` and `remotedaemon_com_consumer_lighthouse` – have been introduced to handle email and Lighthouse data consumption from RabbitMQ queues. These containers are configured with appropriate dependencies and volumes to facilitate seamless operation. This change enhances the system's ability to process asynchronous tasks related to email delivery and website health checks. Several dependency updates were also made, including `@types/node` and `electron-to-chromium`, as well as adjusting the Git repository URL for the client typescript project. These updates ensure compatibility with newer versions of these libraries and provide access to recent bug fixes and improvements. Minor adjustments have been incorporated throughout the codebase, improving code readability through comments and addressing minor inconsistencies in styling. In summary, today's changes represent a strategic advancement towards bolstering user security (password recovery), streamlining data processing (email/lighthouse consumers), and enhancing the overall system’s robustness and maintainability through improved infrastructure and dependency management. These updates lay a solid foundation for future feature development and scalability. Fri, 25 Apr 2025 23:59:59 +0000 Application Updates: New Features & Improvements https://www.codexworker.com/post/127/a What developments have taken place today? Today’s work focused on enhancing user administration features within the application, improving data handling, and streamlining the login process. Key changes include: * **New User Search Form:** A new form has been introduced in the admin panel to allow administrators to filter users based on criteria like email and sorting options. This improves the efficiency of finding specific users within a large user base. * **User Listing Functionality:** A new `UserController` function, `user()`, handles the logic for retrieving and displaying a list of users. It leverages pagination to manage datasets effectively and integrates with the repository layer to access user data. The use of `GetIntQueryParam()` ensures safe retrieval of page and per-page numbers from URL parameters. * **Admin Interface Updates:** Significant improvements have been made to the admin interface, including a new `admin/users.go.tmpl` template that displays user information in a table format with sorting and pagination. A `TemplateFunctions.RenderPagination()` function has also been added for consistent pagination rendering. The Gin router includes a route `/admin/user` to handle these requests. * **Domain URL Updates:** The application's domain configuration has been updated to `http://localhost:18114`, and the header template now dynamically displays a link back to this domain, providing a consistent user experience. * **Website URL History Feature:** A new route `/admin/website/:id/check-certificate` has been added to initiate certificate checks for websites. The `WebsiteUrlRepository` is updated to include functionality for retrieving website URLs and their associated details. The `admin/users.go.tmpl` template now includes the newly generated header. * **Dynamic Login Form:** The login form has been redesigned using JavaScript event handlers, providing a more interactive user experience with visual feedback during submission. The code handles both successful and failed login attempts, displaying appropriate error messages. * **Mock Database for Testing:** A mock implementation of the `WebsiteUrlRepository` (`WebsiteUrlRepositoryMock`) has been created to facilitate testing without relying on a live database connection. This simplifies test development and execution. * **ServerTable Component:** A new `ServerTable` component has been introduced, designed to fetch data from a server using an asynchronous callback function. It includes client-side filtering, sorting, pagination, and error handling. The code refactors existing domain listing functionality to use the new `ServerTable` component. * **Login History Management:** A new endpoint `/admin` has been added to provide access to an administrative interface. This interface allows administrators to manage website URLs and their associated data. Summary: These changes represent a significant step towards improving the administration capabilities of the application, enhancing user experience through dynamic forms, and streamlining key operational processes like certificate management and domain URL retrieval. The introduction of mock database functionality and client-side components promotes testability and performance, while the updates to the admin interface provide administrators with greater control and visibility over the system. Thu, 24 Apr 2025 23:59:59 +0000 Today’s Development Updates: Data Tracking, UI & Real-Time Features https://www.codexworker.com/post/126/a What developments have taken place today? Today’s commits primarily focus on enhancing data tracking, improving UI elements, and introducing new functionality for real-time updates and form handling. Several key changes were implemented: * **Notification Queue:** A dedicated queue has been added to the admin interface for managing notification messages, streamlining the process of sending notifications within the system. * **Dependency Updates:** Numerous npm packages and the `remote-daemon-client-typescript` project have been updated with bug fixes and improvements, ensuring a stable and reliable codebase. Notably, significant changes were made to the `entities` and `parse5` packages. * **RabbitMQ Generator Enhancements:** The `helper_rabbitmq_generator` module has been expanded with new functions (`GenerateMessagesForWebsiteChecker`, `GenerateMessageForLighthouseChecker`, and `GenerateMessageForEmailSending`) to facilitate automated message generation for website monitoring tasks via RabbitMQ, including website health checks and email notifications. This introduces a more structured and efficient approach to triggering external processes. * **Website URL Detail Page Improvements:** The website detail page has been redesigned with breadcrumbs for improved navigation, dynamic title rendering, and JavaScript variable injection. This allows the page to display relevant context and interact effectively with the API. Furthermore, metrics display has been refined to show integer ping durations. * **Live Time Updates:** A system for displaying live timestamps for website URL data has been implemented, providing real-time updates on website health and performance through Server-Sent Events (SSE). This allows users to monitor changes in near real-time. * **Form Structure Refactor:** The login, registration, and user edit forms have been modernized by introducing new `LoginForm` and `RegistrationForm` structs, promoting code reusability and maintainability. The form handling logic has also been streamlined with the introduction of a base `BaseForm` class. * **Redis Pub/Sub Integration:** The system now utilizes Redis Pub/Sub channels to publish domain data updates, enabling real-time synchronization across different components. A new `RedisPubSub` struct is included for managing channel interactions. These changes collectively improve the system's efficiency, user experience, and scalability by introducing robust notification management, dynamic UI elements, real-time data updates, and a more organized codebase. The integration of RabbitMQ and Redis Pub/Sub demonstrates a commitment to leveraging asynchronous messaging patterns for improved performance and responsiveness. Summary: Today’s changes are important because they significantly enhance the system's ability to monitor website health, deliver notifications efficiently, provide a better user experience through dynamic UI elements, and leverage modern technologies like Server-Sent Events and Redis Pub/Sub for real-time data synchronization. These improvements contribute to a more robust, responsive, and maintainable application. Wed, 23 Apr 2025 23:59:59 +0000 Today's Development Highlights: Docker Updates, UI Improvements & Performance Boosts https://www.codexworker.com/post/125/a What developments have taken place today? This commit streamlines the Docker setup for remote daemon services by unifying image creation and entrypoint management. Key improvements include updating the base Docker image to ensure consistency across all services, standardizing entrypoints using a Go build tool (`air`) for faster rebuilds triggered by file system changes, and adding environment variables for monitoring configuration. Furthermore, the commit incorporates Gitlab keys for secure repository access and implements an `inotify-tools` based monitoring loop for automated code deployment upon `.go` file modifications. Temporary SQL scripts have been added to facilitate development and testing with synthetic data. Dependency updates were also applied. Additionally, the commit addresses several UI/UX enhancements: directory creation logic is implemented to prevent errors during file operations, the "ID" column in tables now displays concatenated values with a period separator, and a new user edit page has been introduced, allowing users to update their profile information (email, timezone, locale) via a web interface. The styling of table cells dynamically changes color based on data status using custom CSS classes and icons. Finally, the commit refactors data retrieval from Redis, utilizing aggregated metrics and fixed timestamps for improved API response efficiency and clarity. The code also updates how URL metrics are saved to streamline the process. A new controller has been implemented for handling Server-Sent Events (SSE), enabling real-time domain updates. In summary, these changes significantly improve deployment reliability, maintainability, user experience, and data retrieval performance, creating a more robust and efficient system for managing remote daemon services. Tue, 22 Apr 2025 23:59:59 +0000 System Updates: Stability, Security & User Experience Improvements https://www.codexworker.com/post/124/a What developments have taken place today? Today’s development focuses on enhancing system stability and security through several key changes: improving email processing, streamlining client repository generation, refining registration form handling, optimizing table sorting and pagination, updating dependency versions, and implementing resource limits for services. A critical update involves bolstering email handling within the remote daemon application. New API endpoints have been introduced for sending verification emails and updating user profiles, alongside a redesigned data structure (`UserProfileUpdateRequest`) to facilitate these operations. The email service has been updated with new functions to construct verification emails, utilizing a new `EmailValidationLinkCreator` interface. The registration form has undergone significant refactoring, incorporating error handling improvements using JSON-based responses (specifically the `PostRegistrationBadRequest` error type) and a new template for generating individual form fields. These changes improve user experience and provide more informative feedback during registration. Significant effort was dedicated to simplifying and standardizing client repository generation. New shell scripts (`tag_generated_clients.sh` and `generate_clients.sh`) have been added, automating the process of creating client code from OpenAPI definitions and tagging them with release identifiers. The core logic has also been restructured to leverage a new class, `ApiResponseWebsiteUrl`, for representing API responses, improving modularity and testability. Furthermore, several minor improvements were implemented across different areas: adding resource limits for services using a default CPU allocation, updating the `CustomTable` component's sorting and pagination mechanisms (including improved internationalization support), refining the registration form’s presentation and error handling, updating dependency versions for various libraries, and adjusting routing rules to handle user requests more effectively. Finally, the Docker Compose configuration has been updated to include profiles for different service types, offering greater flexibility in deployment scenarios. In summary, today's changes represent a concerted effort to enhance system reliability, improve developer productivity through automation, strengthen security, and optimize user experience—all contributing to a more robust and maintainable application. Mon, 21 Apr 2025 23:59:59 +0000 docker issues https://www.codexworker.com/post/86/a evening review #13 little bit of refactoring and optimizing docker on localhost, running 10 containers without limits on older laptop caused performance issues... but solved. git diff --shortstat "$(git rev-list -1 --before=midnight HEAD)" HEAD 93 files changed, 5144 insertions(+), 2753 deletions(-) Mon, 21 Apr 2025 18:04:57 +0000 Automatic information https://www.codexworker.com/post/85/a Hello! Code change information from bot here! During week 2025-04-14 - 2025-04-20 i made 127 commits across 4 of my projects. In total i created 22970 new lines and removed 14855 lines. Mon, 21 Apr 2025 00:01:46 +0000 Remotedaemon Project: New Container Services & Performance Updates https://www.codexworker.com/post/123/a What developments have taken place today? Today’s commits introduce several new container services for monitoring and managing various aspects of the `remotedaemon` project, alongside improvements to existing components and data handling. The primary focus is on establishing a robust and automated system for checking consumer health, processing notifications, and tracking website URL metrics. Here's a breakdown of the key changes: * **Consumer Checker Container:** A new Docker container (`remotedaemon_com_consumer_checker`) has been added to continuously monitor and rebuild the consumer application using `air`. It includes SSH access for GitLab integration, automated memory logging, and volume mounts for code, data, and Docker socket communication. * **Notification Consumer Container:** A new container service (`remotedaemon_com_consumer_notification`) handles RabbitMQ notifications, utilizing OpenTelemetry tracing for observability. This allows proactive alerts based on website performance changes. * **Data Writer Container:** A new container service (`remotedaemon_com_consumer_write_data`) is introduced to handle data writing operations, integrating with RabbitMQ and maintaining consistency through database updates. * **Redis Metric Storage:** The codebase has been updated to store website URL metrics in Redis, enabling faster retrieval of performance statistics for API responses. This includes a new `RedisStatSaver` struct and related logic. * **API Response Enhancements:** The `ApiResponseDomain` struct now incorporates a `total_url_count` field to provide more comprehensive information about domain performance. The associated decoding and encoding logic has been updated accordingly. * **Data Update Improvements:** The `CustomTable` component has been enhanced with dynamic data updates, resource cleanup for DOM elements, and new action icons for exporting and deleting domains. SSE integration is included (commented out) to support real-time data updates. * **Subproject Identifier Updates:** Two subproject identifiers have been updated to reflect newer versions of the underlying projects. These changes collectively aim to improve the system's reliability, scalability, observability, and maintainability by introducing dedicated containerized services for key operational tasks and centralizing data storage for efficient access. The use of technologies like `air`, Redis, and RabbitMQ demonstrates a commitment to modern infrastructure practices. Summary: These changes are important because they establish a more robust and automated system for monitoring, managing, and reporting on the health and performance of website URLs within the `remotedaemon` project. They improve operational efficiency, provide better visibility into critical metrics, and lay the foundation for future enhancements such as real-time data updates through SSE. Sun, 20 Apr 2025 23:59:59 +0000 Jaeger Tracing Implementation Enhances Observability https://www.codexworker.com/post/122/a What developments have taken place today? This commit implements a Jaeger-based tracing solution for the application, enhancing observability and debugging capabilities. The core changes involve initializing an OpenTelemetry tracer, configuring it to send data to a Jaeger endpoint, and ensuring proper context propagation throughout the system. Specifically, the `helper_tracer` package was introduced, containing functions to initialize an OTLP tracer using the `go.opentelemetry.io/otel` library. The `InitTracer()` function now establishes a connection to Jaeger via HTTP, utilizing a service name for identification and defaulting to `http://jaeger:4318/v1/traces`. To prevent potential memory exhaustion, a memory limit of 3000 traces has been configured for the Jaeger service. Furthermore, context propagation is implemented using `propagation.NewCompositeTextMapPropagator` to link traces across services and components, ensuring that tracing data accurately reflects request flows. Modifications were also made to the RabbitEnqueuer and its associated methods to accept and utilize a provided context during message queuing. Finally, resource creation with service names was added for improved observability. These improvements are significant because they introduce a foundational layer of tracing, enabling developers to monitor application performance, identify bottlenecks, and troubleshoot issues more effectively through detailed trace data, ultimately leading to increased system stability and reduced debugging time. Sat, 19 Apr 2025 23:59:59 +0000 Website Updates: Monitoring, APIs, & UI Improvements https://www.codexworker.com/post/121/a What developments have taken place today? This commit significantly enhances website monitoring capabilities by introducing a comprehensive “heartbeat” logging system integrated with Redis and an admin interface. It establishes a structured approach to tracking service availability, providing administrators with immediate insights into potential issues through the admin panel, which displays heartbeat status and consumer information from RabbitMQ queues prefixed with "remotedaemon_". Furthermore, it emphasizes the importance of metrics for long-term application health by detailing a website checker that measures message insertion to data writing/notification delivery times. This commit also refines the API endpoints related to website URL metrics. The `ApiHandlerWebsiteUrl` struct has been updated to include a more complete representation of website URLs, incorporating data from the associated `website_url` table and adding validation for average ping duration. The code now utilizes this enriched information in API responses, providing more context for monitoring purposes. Additionally, the commit enhances database schema by introducing a `website_id` column into the `website_url_ping_history` table, aligning with changes made to repository queries. Another key development involves enhancing the website detail page user experience. The template and associated JavaScript have been updated to display the ID of the viewed website URL, improving usability and providing clearer identification for each entry. The previous date-based navigation and push notification links have been removed in favor of this more direct approach. A new script tag dynamically loads specific JavaScript files based on the website’s ID, ensuring targeted functionality. Further improvements include modifications to the website listing page, adding a button for quickly adding new URLs associated with the currently displayed website. The addition of a `<script>` tag and a corresponding button streamlines the URL management process, simplifying user interaction. Finally, this commit also refactors several repository classes, introducing data validation in `ApiHandlerDomain` and enhancing the functionality of the `MockWebsiteUrlMetricRepository`. In summary, today's changes are important because they bolster website monitoring, refine API data delivery, enhance user interface elements on the detail page and listing page, and improve code organization for maintainability. These combined efforts result in a more robust, informative, and user-friendly application experience. Fri, 18 Apr 2025 23:59:59 +0000 evening review #10 https://www.codexworker.com/post/84/a evening review #10 little bit of refactoring, tests and thinking about rabbit queues... git diff --shortstat "$(git rev-list -1 --before=midnight HEAD)" HEAD 70 files changed, 4685 insertions(+), 697 deletions(-) Fri, 18 Apr 2025 14:42:37 +0000 Today's Updates: Firebase, Email, Pings & Monitoring Improvements https://www.codexworker.com/post/120/a What developments have taken place today? Today’s commits focus on streamlining Firebase messaging, enhancing email data handling, refining website ping tracking, and introducing new monitoring capabilities. The primary changes involve decoupling message sending from a generic FirebaseApp interface, centralizing email logic for improved clarity, consolidating website URL ping history data management, adding real-time metrics dashboards with RabbitMQ integration, and updating Docker image saving procedures. Specifically, the first commit refactors the Firebase Cloud Messaging (FCM) sender by introducing a `MessagingClientInterface` dependency injection mechanism. This allows the `FCMSender` struct to utilize an external client for message sending, enhancing flexibility and testability. The accompanying mock implementation (`MockMessagingClient`) facilitates unit testing of this new approach. The changes also updated the test cases to verify correct message delivery and error handling through the injected client. The second set of changes concentrates on email data structures and service calls. A `DataForUserRegistrationEmail` struct has been introduced, consolidating user-related information for registration emails. Similarly, a `DataForPingFailedEmail` struct is now used for sending website ping failure notifications, complete with methods to determine template IDs and generate variable maps for email rendering. The `SendRegistrationEmail` and `SendWebsitePingFailed` functions have been updated to utilize these new structs, centralizing email sending logic within the `sendEmail` helper function. This function now handles database storage and RabbitMQ queuing of emails. Next, the commit refactoring website ping history data focuses on optimizing database queries for retrieving historical data. Instead of fetching all matching rows and selecting the first one, a dedicated `SearchOne` function is used to directly query for a single record based on search criteria, reducing unnecessary data retrieval. Error handling has also been added in case no records are found, preventing potential errors during data access. Finally, a new `WebsiteURLPingHistorySearch` function simplifies search parameters and limits results to one record. Redis integration is updated to prevent issues when retrieving data for newly pinged URLs. Further changes include updating the Docker image saving process, removing unused code, and enhancing the admin dashboard with real-time information on RabbitMQ queues and heartbeat status for various services. This includes adding a `helper_heartbeat` module, new endpoints `/admin/heartbeats` to retrieve heartbeats, color-coded table visualization of service health, and Redis integration via `github.com/go-redis/redis/v8`. Finally, the commit introduces a new test suite for file encryption and decryption using AES, along with updates to package versions and minor code cleanup. These changes collectively contribute to improved code organization, maintainability, and functionality across various aspects of the application. In summary, these changes represent significant improvements by introducing structured data handling, decoupling message sending, optimizing database queries, enhancing monitoring capabilities, streamlining Docker image management, and improving overall test coverage – ultimately leading to a more robust and efficient system. Thu, 17 Apr 2025 23:59:59 +0000 Today’s Development Updates: Key Changes Across Subprojects https://www.codexworker.com/post/119/a What developments have taken place today? Today’s commits primarily focus on improving data management, streamlining processes, and enhancing code organization across several subprojects. These changes aim to increase modularity, improve performance, and enhance the application's overall stability and maintainability. **Subproject Identifier Updates:** Two subproject identifiers were updated with newer commit hashes, ensuring consistent references within the codebase. This is a routine maintenance task to keep dependencies synchronized. **Repository Login History & Domain Metadata (New Packages):** Significant changes were made to introduce new packages for managing login history and domain metadata. These packages include: * `LoginHistory` and `LoginHistoryRepository`: Structures for representing login data and a repository for database interaction, including methods for saving, searching with sorting capabilities, and retrieving single entries. * `DomainMetadata` and `DomainMetadataRepository`: Structures for representing domain metadata and a corresponding repository with similar functionalities to the login history package. * The `Registry` struct was updated to use these new repositories, establishing a more organized data structure. **TypeScript Client Project Improvements:** A new hook was added to TypeScript client projects to include a necessary `package.json` file for tooling and dependency management. This ensures the clients have the foundational elements needed for development. **Website URL Handling (Enhanced Validation & Display):** The code now enforces stricter validation of website URLs, rejecting those starting with "http://" or "https://". Additionally, a new table component (`CustomTable`) was created to display and manage website URLs, with added functionalities such as sorting, adding, and removing entries. The `renderDomainTable` function now efficiently renders this table. **Email Service Refactoring (RabbitMQ Integration & Validation Links):** The email service architecture has been significantly revamped. It now utilizes interfaces for email validation link creation, saving emails to the database, and enqueueing messages to RabbitMQ. This promotes modularity and simplifies asynchronous email delivery. **Certificate Management (Database Updates & Error Handling):** The codebase was updated to handle domain and certificate errors within the database. New fields were added to the `Website` table to store these error details, enabling more detailed reporting and debugging. **Dependency Upgrades:** Several npm packages, including Chart.js and regjsparser, were upgraded to their latest versions, improving stability and incorporating new features. Additionally minor updates for other dependencies were included as part of routine maintenance. These changes are important because they address fundamental areas of the application – data management, client tooling, communication processes, and dependency management – leading to a more robust, maintainable, and efficient codebase overall. Wed, 16 Apr 2025 23:59:59 +0000 golang interfaces https://www.codexworker.com/post/82/a evening review #8 Golang interfaces are nice, i think i started to think a little bit in golang way git diff --shortstat "$(git rev-list -1 --before=midnight HEAD)" HEAD 119 files changed, 3227 insertions(+), 1805 deletions(-) #golang #buildinpublic Wed, 16 Apr 2025 16:30:44 +0000 Key Developments: Updates to Client, Security, and Deployment https://www.codexworker.com/post/118/a What developments have taken place today? Today’s commits introduce several key additions and improvements across various parts of the project. A new Dart client submodule is added for a remote daemon client, mirroring existing TypeScript functionality. This includes initializing a Git connection and updating configuration files. A `WebsiteURLAddForm` class has been introduced to provide a modal dialog for adding website URLs with validation and API integration via the `websiteUrlApi`. Significantly, a major overhaul of access control was implemented, shifting from an ad-hoc system to a structured approach using the new `acl` package. This involved defining roles, resources, actions, and implementing functions like `isLoggedIn`, `ownsDevice`, and `allowAll`. The access control logic was then refactored in several controller methods, leveraging these definitions for authorization checks. Furthermore, unit tests were updated to reflect these changes. A new endpoint for deleting domains (`DELETE /api/v1/domain/{id}`) has been added with comprehensive error handling and security features utilizing the `XAuthorization` mechanism. Finally, a script was implemented to tag multiple client projects with specified version numbers, streamlining the process of deploying updates across different projects. Summary: These changes represent important steps towards improving code maintainability, enhancing API functionality, bolstering security, and simplifying deployment processes. The addition of the Dart submodule expands client support, while the access control overhaul provides a more robust and scalable system for managing user permissions. The new endpoint and related improvements add to the API's capabilities, and the tagging script streamlines project updates. Tue, 15 Apr 2025 23:59:59 +0000 API Updates: Authentication, Error Handling & Documentation Improvements https://www.codexworker.com/post/117/a What developments have taken place today? Today’s work primarily focused on refining the API handler for user authentication and registration, enhancing error handling, and improving documentation through OpenAPI specification generation. Key changes include: * **Login & Registration API Updates:** A new `ApiHandlerRegistration` struct was introduced alongside a `NewApiHandlerRegistration` constructor, enabling full-fledged user registration functionality with email validation and token creation. The `/login` endpoint now handles both login and authentication using the newly implemented token repository. * **Robust Error Handling:** Significant improvements were made to error handling across multiple components. This involved defining new error types (`PostLoginUnauthorized`, `PostRegistrationInternalServerError`), implementing proper JSON encoding for these errors, and enhancing response handling in various handlers to provide more informative feedback to clients. * **OpenAPI Documentation Generation:** The code now leverages the `ogen` tool to automatically generate OpenAPI specifications from Go source code. This streamlines API documentation maintenance and ensures consistency between the codebase and the API definition. * **Dependency Updates:** Several dependencies were updated, including those related to error handling, JSON encoding, and cryptographic functions, improving security and functionality. In essence, these changes collectively enhance the robustness, clarity, and maintainability of the user authentication and registration processes within the application, providing a more reliable and well-documented API for clients to interact with. The improvements in error handling are particularly important for ensuring stability and providing useful feedback to users. Mon, 14 Apr 2025 23:59:59 +0000 ogen https://www.codexworker.com/post/80/a evening review #6 Started ogen from scratch today. Weird at first, but type-safe and clean. #golang #openapi #ogen #buildinpublic Mon, 14 Apr 2025 16:31:58 +0000 Automatic information https://www.codexworker.com/post/79/a Hello! Code change information from bot here! During week 2025-04-07 - 2025-04-13 i made 139 commits across 5 of my projects. In total i created 19973 new lines and removed 31077 lines. Mon, 14 Apr 2025 00:01:50 +0000 Domain Listing API Enhancements & New Website Additions https://www.codexworker.com/post/116/a What developments have taken place today? Today’s commits have significantly enhanced the domain listing API and associated UI components. The core changes focus on improving data retrieval, filtering, sorting, and display – all geared towards a more efficient and user-friendly experience. Here's a breakdown of the key updates: * **API Refactor & Repository Integration:** The domain listing API now utilizes a dedicated `WebsiteRepository` for accessing website data, moving away from direct database queries. This improves modularity and testability. * **Query Parameter Support:** Comprehensive query parameters – `page`, `pageSize`, `search`, and `sort` – have been implemented, allowing users to dynamically filter and sort the list of domains. Pagination ensures efficient handling of large datasets. * **Domain Metadata Improvements:** The database schema has been updated to include a unique constraint on domain names and improved data types for certificate information. A new helper package (`helper_domain`) handles WHOIS lookups for retrieving detailed domain information. * **Certificate Management Enhancements:** Certificate retrieval, validation, and storage are now more robust with the introduction of a `Dialer` interface for network connections and better error handling. The `Domain` struct now incorporates key usage and SAN validation. * **Table Styling & Accessibility:** The table component's styling has been refined for improved visual appeal and accessibility – including responsive design, sorting indicators, and enhanced link appearance. * **API Documentation Generation:** A new endpoint (`/api-ts`) generates TypeScript definitions for API routes, facilitating integration with other projects. * **New Website Addition**: Three websites have been added to the database through an automated process. In summary, these changes represent a significant step toward consolidating domain management functionality and providing a more robust, flexible, and user-friendly experience for interacting with website data. The focus on modularity, improved data validation, enhanced UI elements, and streamlined API interactions underscores a commitment to maintainability and scalability. Sun, 13 Apr 2025 23:59:59 +0000 RabbitMQ Architecture Restructuring & Key Updates https://www.codexworker.com/post/115/a What developments have taken place today? Today’s work focused on significantly restructuring the RabbitMQ consumer architecture for increased modularity, testability, and maintainability. Key changes include: * **Generic Consumer Pattern:** A new `GenericConsumer` struct and function were introduced to standardize message handling across different RabbitMQ message types (email, checker, certificate). This allows for flexible message processing through a registry object and handler interface. * **RabbitMQ Helper Package:** The core RabbitMQ logic was moved into the `helper_rabbitmq` package, promoting separation of concerns with new functions like `NewRabbitMQ`, `QueueMessage`, and interfaces such as `RabbitConsumerDataEmail`. * **Specific Consumer Structs:** Separate consumer structs (`CertificateConsumer`, `CheckerConsumer`, `EmailConsumer`) were created for each message type, inheriting from the `GenericConsumer` and utilizing the registry object to access relevant data. * **JSON Message Serialization:** Functions were added to marshal and unmarshal RabbitMQ messages into JSON format, facilitating easier communication between components. * **Refactoring of Existing Functions:** The `CheckLightHouse` and `InsertWebsiteUrlToRabbitMQ` functions were refactored to utilize the new generic consumer pattern and `QueueMessage` method respectively. Email verification and registration handlers now leverage the new consumer functions for message publishing. * **Configuration Updates:** The RabbitMQ queue naming conventions were updated in the configuration file (e.g., `checker`, `lighthouse`, `email`, `certificate`). * **Enhanced Admin Dashboard:** The admin dashboard was updated with a loading indicator and placeholder element to provide visual feedback during data fetching, improving user experience. This commit makes several changes related to RabbitMQ integration within the application. Here's a breakdown of the modifications: 1. **RabbitMQ Configuration Updates:** The `Config` struct now includes fields for various queues under the `"rabbit_queues"` section, centralizing queue names in configuration and the `RabbitMQ` struct accepts a RabbitMQ address during initialization. 2. **`NewRabbitMQ` Function:** A new function `NewRabbitMQ` simplifies RabbitMQ instance creation by taking the RabbitMQ address as an argument, promoting code reusability. 3. **`SendMessageToRabbitMQ` Function Updates:** The `SendMessageToRabbitMQ` function now takes a queue name and message body as inputs, making it more versatile. It uses the provided queue name to declare and publish messages to the correct RabbitMQ queues, with improved error handling via logging. 4. **`InsertWebsiteUrlToRabbitMQ` Function:** This function remains largely unchanged but now leverages the `NewRabbitMQ` function for RabbitMQ instance creation, improving code clarity. It also uses the correct queue name from configuration. 5. **Changes in Consumer Logic (Checker & Lighthouse):** The `StartCheckerConsumer` and `CheckLightHouse` functions were updated to use the new `NewRabbitMQ` function to instantiate RabbitMQ clients and send messages to appropriate queues based on configuration settings. 6. **Email Verification and Registration Handler Changes:** The `StartCertificateC` function was updated to utilize the new RabbitMQ consumer structure and message publishing logic when sending email verification messages. This commit modifies the code related to displaying website expiration information in a table. Here's a breakdown of the changes: * **API Response Types:** The `WebsiteCertificate` struct is updated to allow for optional (nullable) string values for expiration dates and certification expiration dates, replacing them with pointers to `time.Time`. This accommodates cases where an expiration date might be unavailable or not yet set. * **API Response Data:** The `ApiResponseDomainExpiration` struct is adjusted to accommodate the new nullable time types. It now uses `string | null` for `expiration_date` and `certification_expiration_date`, and includes logic within the constructor to format these times into RFC3339 strings when available, otherwise leaving them blank. * **Database Model:** The `Website` struct is updated to use `sql.NullTime` instead of `sql.NullString` for `CertificationExpiration`. * **Database Save Operation:** The database save operation now uses `UTC_TIMESTAMP()` instead of the previous timestamp method. * **Rendering Row Logic**: Changes were made to rendering the rows in the table, including using conditional logic and CSS classes based on the expiration dates and days remaining to expire, to visually indicate certification status (green, orange, red). This commit refactors the code to improve certificate expiration handling within the CertificateConsumer. Specifically, it replaces the `helper_domain.NewDomain` function with a new one that takes a `WhoisClient` and `CertificateFetcher` as arguments. The `NewDomain` function now injects Sat, 12 Apr 2025 23:59:59 +0000 Image Handling & Core Updates: A Significant Commit https://www.codexworker.com/post/114/a What developments have taken place today? This commit significantly enhances image handling within the application, introducing new functionality for resizing, uploading, and displaying images. Key changes include: * **Configuration Structure Updates:** The `Config` struct now includes fields for storing the paths to directories where resized images are saved (`data_resized_directory`) and original files are stored (`data_directory`). * **Image Resizing Logic Implementation:** A new `GenerateImageDiskPath` function is implemented, generating unique filenames based on width, height, and quality. It also utilizes the `convert` command-line tool for padding and centering images during resizing, ensuring consistent image dimensions and appearance. Debugging information related to image size and quality has been added. * **File Upload Support:** The `post_add.go.tmpl` template now supports file uploads, allowing users to submit images alongside their posts. * **File Repository Modifications:** New methods (`Save`, `LoadOne`, `UpdatePath`) are added to the `PostFileRepository` struct for managing uploaded files associated with posts. * **Homepage Template Updates:** The homepage template is updated to display a list of uploaded images linked to each post, utilizing image paths obtained from the file repository. * **PostAddValidator Enhancements:** The `PostAddValidator` now validates file uploads, checking for valid formats (PNG, JPG, JPEG) and sizes. It also ensures that required fields like title and content are not empty. * **Fullscreen Modal Implementation**: A new feature has been added which allows users to view images in a fullscreen modal window with larger image sizes (800x800). The layout dynamically adjusts for smaller screens, displaying the image and post text side-by-side. This commit also addresses several technical improvements: * **GitLab Integration Enhancements:** The code now retrieves post statistics from GitLab using the API. * **Timezone Handling Improvements:** Database queries are updated to use UTC\_TIMESTAMP() for consistent time representation across servers and environments. A new `timezone_offset` field is added to the user struct for storing timezone information directly within the database. * **Cron Job Setup**: A cron job has been created to run the `command/cron` binary, responsible for scheduling tasks within the application. * **Secret Management Enhancements:** The system now encrypts sensitive configuration data using AES-256-CBC with PBKDF2 key derivation, improving security and simplifying key management. These changes collectively provide a more robust, user-friendly, and secure experience for managing images and related content within the application. Summary: This commit significantly expanded image handling capabilities by incorporating resizing logic, file upload support, and improved display features. It also implemented enhanced security measures through secret encryption, streamlined repository management, and addressed technical improvements related to GitLab integration and timezone consistency, resulting in a more functional and reliable user experience. Fri, 11 Apr 2025 23:59:59 +0000 Today's App Updates: Significant Improvements Across Image Handling, User Experience & More https://www.codexworker.com/post/103/a What developments have taken place today? Several significant improvements and additions have been implemented across various parts of the application, primarily focused on image handling, user interaction, and website display enhancements. These include: 1. **Configuration Updates:** The `Config` struct now includes fields for storing the paths to directories where resized images are saved (`data_resized_directory`) and original files are stored (`data_directory`). This facilitates image URL generation and file organization. 2. **Image Resizing & Uploads:** A comprehensive image resizing system has been implemented, utilizing the `convert` command-line tool for padding, centering, and compression. The `GenerateImageDiskPath` function now dynamically constructs unique filenames based on width, height, and quality parameters. File upload functionality is also integrated into the ‘Post Add’ form, enabling users to submit images alongside their posts. 3. **File Repository Enhancements:** New methods (`Save`, `LoadOne`, `UpdatePath`) have been added to the `PostFileRepository` struct for managing associated files within posts. 4. **Homepage & Post Display Updates:** The homepage template now displays uploaded image paths, and the post display has been revamped to support larger images (800x800) alongside smaller thumbnails, utilizing a fullscreen modal view that adapts to different screen sizes. Responsive design adjustments ensure proper layout on smaller screens. 5. **GitLab Integration:** Code has been added to retrieve post statistics from GitLab using the GitLab API and handle commit information retrieval. The `GitLabReader` struct now includes functionality for fetching all files associated with specific posts. 6. **Newsletter Subscription:** The homepage’s newsletter subscription process is now handled asynchronously, validating email format and preventing duplicate submissions through a series of checks and event listeners. 7. **Testing Framework Improvements:** The testing suite has been updated to use `npm run test` instead of `npx jest`, with enhanced error handling and build aborting on test failure. It also incorporates mocking of the `Date.now()` function for consistent test results. 8. **Time Difference Display:** A new JavaScript file (`/js/post.js`) has been added to display human-readable time differences for posts, leveraging a `humanReadableTimeDifference` function and a `pluralize` helper. The existing `homepage.js` was updated to reference this new file. 9. **Modal Closure:** The application now properly closes modals when the Escape key is pressed. 10. **Validation Updates:** The `PostAddValidator` has been enhanced to include validation for file uploads (size and format) and to ensure that required fields like title and content are not empty. Additionally, it filters out empty files during validation. 11. **Image Resizing Logic Refinement:** A dedicated JavaScript file (`/js/post.js`) handles the resizing of images using a new function, accommodating a 100x100 pixel size with an 80-quality setting and includes debugging logging for supported sizes. The `Convert` function now incorporates additional arguments to improve image compatibility. 12. **Form Submission Logging:** Added logging within the form submission handler to print form data to the console, aiding in debugging. In summary, these changes significantly enhance the application’s functionality by enabling rich media support (images), streamlining user workflows (file uploads and newsletter subscriptions), improving website presentation (post display and responsive design), and strengthening code quality through testing improvements and validation rules. The integration of GitLab statistics provides valuable data insights, while the modularized JavaScript files promote maintainability and extensibility. Fri, 11 Apr 2025 23:59:59 +0000 Today’s Updates: Image Handling & GitLab Improvements https://www.codexworker.com/post/94/a What developments have taken place today? Here’s a summary of the commits, outlining their improvements: * **Configuration Enhancements:** The codebase now includes `data_resized_directory` and `data_directory` within the Config struct to manage image storage paths effectively. This improves organization and URL generation for resized images. * **File Upload Functionality:** A file upload field has been added to the `post_add.go.tmpl`, enabling users to submit images alongside their posts, expanding content creation possibilities. * **Post File Repository Updates:** The `PostFileRepository` struct now supports saving, loading, and updating file paths associated with posts, providing a structured way to manage post-related files. * **Homepage Template Update:** The homepage template has been updated to display uploaded images linked to each post using the new image path information, enhancing visual presentation. * **GitLab Integration Improvements:** Code now retrieves post statistics from GitLab and correctly handles commit information retrieval via the GitLab API, streamlining data synchronization. * **GitLab Reader Enhancement:** The `GitLabReader` struct incorporates functionality to fetch all files associated with specific posts, leveraging the updated file repository. * **Frontend Template Updates:** The header template now includes img tags for displaying images related to each post, and a new modal view is implemented providing a fullscreen experience for larger images and improved responsiveness across different screen sizes. * **Image Resizing Logic:** Implemented logic using the `convert` command-line tool for resizing images with specified dimensions, quality, padding/centering, and color space settings – ensuring unique filenames and optimized image quality. * **Validator Enhancement:** The `PostAddValidator` has been updated to validate file uploads (size, format), title, and content, strengthening data integrity. * **Test Suite Improvements:** A new test suite for the `humanReadableTimeDifference` function has been added, using Jest and mocking `Date.now()` for consistent testing results; a build process check ensures tests pass before building. Additionally, the build process now utilizes `npm run test`. * **Newsletter Subscription Updates:** The newsletter subscription functionality on the homepage is updated to handle email validation, prevent duplicate submissions, and provide user feedback through informative messages and button states. * **Post Display Refinement**: Introduced a "post-wrapper" div that stacks text content and images vertically for improved layout, along with responsive adjustments for smaller screens. These changes collectively create a more robust and user-friendly application by adding image handling capabilities, improving data management, enhancing the user interface, and strengthening data validation processes. The improvements to GitLab integration and test automation further enhance reliability and maintainability. The newsletter functionality ensures effective subscriber capture while providing a smooth user experience. In summary, this commit cycle focused on significantly expanding the application's core functionalities – specifically image handling and content presentation – alongside crucial improvements in data management, testing practices, and user interface polish. These changes represent a substantial step forward in enhancing the overall usability and reliability of the system. Fri, 11 Apr 2025 23:59:59 +0000 mariadb ?parseTime=true https://www.codexworker.com/post/74/a Using ?parseTime=true with the Go MySQL driver could help by automatically converting DATETIME and TIMESTAMP fields into Go’s time.Time, making it easier to work with dates and times. #golang #database #mariadb Fri, 11 Apr 2025 07:39:52 +0000 evening review #2 https://www.codexworker.com/post/73/a evening review #2 Two bots chatting tests: hw: docker container: 1 cpu, DDR3 memory cpu: Intel(R) Core(TM) i7-5600U CPU @ 2.60GHz result: Connections established: 2 Messages received: 83906 Errors encountered: 0 Time interval: 60s #golang #websocket Fri, 11 Apr 2025 06:41:54 +0000 Today’s Development Updates: Stability & Improvements https://www.codexworker.com/post/113/a What developments have taken place today? Today's commits focused on enhancing WebSocket server stability, improving API controller integration with a registry, refining database interactions, and adding logging functionality. Several changes were made to ensure robust error handling, thread safety, and debugging capabilities throughout the application. Specifically: * **WebSocket Server Enhancements:** A new context cancellation mechanism was implemented for user connections, enabling graceful shutdown and preventing resource leaks. The WebSocket server now includes improved logging to indicate errors during message sending and error responses, aiding in debugging. Ping intervals are handled more gracefully within the connection context. * **API Controller Updates:** API controllers were updated to utilize a `registry` for accessing external services (file repositories). Constructors now accept the registry as an argument, improving modularity and testability. Database queries were adjusted to include more parameters and logging statements for improved performance and data integrity. * **Database Improvements:** The `UserChannelRepository` was modified to include a function (`GetOtherUserFileId`) that retrieves file IDs associated with other users in one-on-one channels, facilitating channel assignment logic. Database queries were updated to include more parameters and logging statements for better performance and debugging. * **Logging Enhancements:** The server now integrates with a Redis database for persistent log storage, including caller information (file name and line number) to aid debugging. Existing direct logging calls have been replaced with the new Redis-based approach. * **Test Improvements:** A statistics tracking system was added to WebSocket tests, monitoring connections, messages sent, and errors encountered. These changes collectively contribute to a more stable, reliable, and maintainable application by addressing critical areas such as error handling, data consistency, and debugging support. The improvements ensure graceful connection management and facilitate efficient resource utilization within the WebSocket server. Thu, 10 Apr 2025 23:59:59 +0000 WebSocket Server Stability & Debugging Enhancements https://www.codexworker.com/post/102/a What developments have taken place today? Today’s commits focus on enhancing WebSocket server stability, reliability, and debugging capabilities across several areas: error handling, connection management, logging, and test statistics tracking. Specifically, there were updates to handle errors gracefully during message sending, implement a context cancellation mechanism for user connections, improve the Redis-based logging system, and add a file identifier to channel creation. The code also incorporates a new `RawConnection` struct for comprehensive WebSocket connection tracking, along with improved error handling within the `UserConnection` goroutine. Further enhancements include updating API controllers to utilize a registry for external services, improving database interaction through query adjustments and logging, and adding resource monitoring and plotting functionality to the WebSocket stress test. Finally, there were updates to the repository package for consistent logging, and an overall effort to improve debuggability by including more informative log messages throughout the codebase. Summary: These changes collectively increase server resilience, provide detailed debugging information, and introduce a more sophisticated system for managing channel files, ultimately leading to improved stability and maintainability of the WebSocket backend services. Thu, 10 Apr 2025 23:59:59 +0000 WebSocket Server Updates: Key Improvements & Changes https://www.codexworker.com/post/112/a What developments have taken place today? Here’s a summary of the commit descriptions, outlining the changes and their improvements: **WebSocket Server Improvements:** * **Message Structure Enhancement (v1):** The structure of messages transmitted over the WebSocket server was modified to nest message content within a key-value pair object (“message” key). This improves handling and processing on the receiving end by providing more organized data. * **Connection Management Refactoring (v2):** A new `UserConnection` struct was introduced, along with associated functions (`AddUserConnection`, `RemoveUserConnection`, `AddChannelToUser`, `RemoveChannelFromUser`, `sendErrorToUser`). These changes establish a robust system for managing user connections and channel subscriptions within the server state, ensuring thread safety through mutexes. * **WebSocket Command Structure (v3):** A comprehensive refactoring of the WebSocket command structure was implemented, introducing constants (`Redis_Chat_Command_*`) and structs (`WSCommand`, `WSAuth`). New functions (`NewWSNewMessage`, `NewWSMessageSeen`, `NewWSMessageDelivered`) were created to facilitate validation and JSON serialization/deserialization of commands. This significantly improves maintainability and testability, paving the way for a more structured approach to data exchange. * **Bot Interaction Simulation (v4):** A new test case (`TestTwoBotsChat`) was added, simulating two bots exchanging messages over the WebSocket server. This includes database initialization, bot authentication, channel subscription, message sending/receiving, and verification of routing and delivery. The introduction of helper function `waitForMessage` further enhances testing capabilities. **Authentication & Flow Simplification (v5):** * **Simplified Authentication:** Unnecessary error handling code was removed following successful initial authentication attempts, streamlining the authentication flow and improving efficiency. **Testing & Simulation (v6):** * **Comprehensive WebSocket Testing:** The test function now includes more thorough checks for message handling within a websocket server – subscribing to channels and verifying transmitted/received content. **Utility Script (v7):** * **WebSocket Activity Simulator:** A bash script was created to simulate WebSocket activity, allowing developers to easily test the server’s behavior with random user actions and message sending. The script uses `websocat` for realistic simulation. **Analytics Removal (v8):** * **Analytics Code Deprecation:** Several files related to tracking and analytics – `message.ts`, `sse.ts`, and `analytics.js` – were removed, indicating a shift away from extensive user behavior tracking. **Build & Dependency Updates (v9):** * **Dockerfile Improvements:** The Dockerfile was updated to utilize a slimmer Python image and install the `websocket-client` library. Additionally, specific versions of nvm were enforced within the Dockerfile for consistency. A javascript function was added to ensure proper rendering channel functionality. **Minor Updates (v10 & v11):** * **.gitignore Update:** The .gitignore file was updated to exclude the .gitignore file itself from version control. * **General Code Update:** A general code update occurred, encompassing various bug fixes, improvements and updates without specifying particular details. In essence, these changes represent a significant step forward in improving the WebSocket server’s architecture, functionality, and testability. The refactoring of command handling, coupled with enhanced connection management and bot simulation, lays a solid foundation for future development and ensures reliable message exchange within the system. Removing analytics code indicates a focused approach to core functionality while simplifying the codebase. Wed, 09 Apr 2025 23:59:59 +0000 WebSocket Server Updates: Key Improvements & New Features https://www.codexworker.com/post/101/a What developments have taken place today? Here’s a summary of the commit descriptions, focusing on improvements and key changes: **WebSocket Server Enhancements:** * **Message Structure Refinement (v1):** The structure of messages transmitted over WebSockets has been updated to use a nested object format with “message” as the key. This improves how receiving systems can process the message content, likely providing more structured data for handling. * **User Connection Management:** A new `UserConnection` struct and associated functions (`AddUserConnection`, `RemoveUserConnection`, `AddChannelToUser`, `RemoveChannelFromUser`, `sendErrorToUser`) have been introduced to manage user connections, channel subscriptions, and error reporting within the server’s state. These additions enhance thread safety and provide a more robust mechanism for managing user interactions. * **WebSocket Command System:** A comprehensive WebSocket command system has been implemented utilizing a `WSCommand` base struct and derived structs (`WSAuth`, `NewWSNewMessage`, `NewWSMessageSeen`, `NewWSMessageDelivered`). This refactoring improves code organization, testability, and facilitates the handling of various WebSocket commands. JSON marshaling/unmarshalling functions have also been added to support data exchange. * **Bot Interaction Simulation:** A new test case (`TestTwoBotsChat`) simulates bot interactions over the WebSocket server, verifying message routing and delivery between bots. This ensures the core functionality for bot communication is working correctly. * **Redis Channel Broadcasting:** The system now utilizes Redis channels to broadcast messages to connected clients via `PushToRedisChannel`. This provides a scalable method for distributing information across the server’s user base. **Authentication Improvements:** * **Simplified Authentication Flow:** An unnecessary error handling block after successful authentication has been removed, streamlining the authentication process and improving efficiency. **Testing & Simulation:** * **Enhanced WebSocket Test Functionality:** The test function now includes more thorough checks for message handling during websocket communication. * **WebSocket Activity Simulator Script:** A bash script has been added to simulate WebSocket activity by randomly sending messages or simulating user behavior, allowing developers to easily test the server's functionality in a controlled environment. **Code Cleanup & Refactoring:** * **Analytics Removal:** Code related to tracking and analytics (message handling, SSE, analytics.js) has been removed, suggesting a shift in focus away from extensive data collection. * **.gitignore Update:** The `.gitignore` file has been updated to exclude itself, preventing accidental commits of the ignore file. * **Build & Dependency Updates:** The project's build process and dependencies have been upgraded (NVM, Dockerfile). **Important Changes Summary:** These changes represent a significant upgrade to the WebSocket server’s architecture and functionality. The introduction of structured message handling, robust user connection management, and a centralized command system greatly improve maintainability, testability, and scalability. The addition of the simulation script is a valuable tool for development and testing. Furthermore, simplifying the authentication flow and removing the analytics code demonstrates a focus on streamlining the core server functionality while reducing unnecessary complexity. Wed, 09 Apr 2025 23:59:59 +0000 evening review #1 https://www.codexworker.com/post/72/a evening review #1 [1] simulating multiple websocket connections which need to keep alive is not working well with curl, wscat, websocat. Python is the way [2] working on 14" laptop is really best working setup ... for work Wed, 09 Apr 2025 20:31:55 +0000 Centralized Logging & System Improvements Today https://www.codexworker.com/post/111/a What developments have taken place today? This commit implements a centralized logging system utilizing Redis for enhanced monitoring and debugging capabilities. A new `RedisWriter` struct is introduced to publish log messages to a dedicated "partnerup_zone_chat" channel, alongside a redesigned `Logger` struct that integrates both standard logging and Redis output. The previous `RedisLogger` package has been removed as part of this consolidation. The code now also includes more descriptive constants for message types, improving readability and maintainability, along with enhanced error handling within the `InitWebsocketAuthenticate` function. Specifically, JSON parsing errors are logged and input validation is implemented to ensure data integrity before processing new messages. The marshaling process has been updated to convert JSON structures into byte arrays suitable for Redis publishing. Further improvements include logging enhancements within the message handling logic, specifically tracking incoming websocket messages, message broadcasting, and message processing events using a dedicated `redisLogger`. Image handling has been refined by replacing file extensions with `.webp` and utilizing an `ImageLink` function to generate URLs, defaulting to a predefined image if necessary. Finally, minor HTML adjustments have been made to display message counts in the user interface, alongside fixes for copy-to-clipboard functionality and curl command generation. These changes are important because they establish a more robust, centralized logging infrastructure that provides richer debugging information, improves code clarity through standardized constants and descriptive variable names, enhances error handling for improved stability, and streamlines image processing for enhanced user experience – all contributing to overall system reliability and maintainability. Tue, 08 Apr 2025 23:59:59 +0000 Code Improvements: Logging, Data Handling & Web App Updates https://www.codexworker.com/post/100/a What developments have taken place today? The codebase has undergone significant improvements focused on logging, data handling, and web application configuration. The core of these changes revolves around migrating from a previous Redis logger implementation to a new system utilizing Redis for enhanced logging capabilities. Specifically, a `RedisWriter` struct is introduced to publish log messages to a dedicated "partnerup_zone_chat" channel, alongside a centralized `Logger` struct that manages both standard output and Redis logging for informational, error, and debug levels. The original `RedisLogger` package has been removed. Code clarity and maintainability have been increased by replacing generic string constants with more descriptive alternatives for message types (e.g., `MessageTypeText`, `MessageTypeImage`). Furthermore, the `InitWebsocketAuthenticate` function now includes robust error handling, logging JSON parsing errors, validating input parameters, and converting JSON structures to byte arrays for Redis publishing. The localhost configuration has been updated to include detailed logging within the message handling logic, including tracking new messages received via WebSocket, broadcasting of messages, and processing of incoming messages. Image handling has been optimized by replacing file extensions with `.webp` and generating URLs using `ImageLink`, defaulting to a specified image if needed. Minor HTML modifications have been made to display message counts in the user interface, alongside fixes for copy-to-clipboard functionality and curl command generation. These changes represent a vital step towards improved system observability through centralized logging, enhanced error detection and debugging capabilities within the web application, and streamlined data management practices—resulting in greater stability and easier troubleshooting. Tue, 08 Apr 2025 23:59:59 +0000 Today's Development Updates: Analytics, Domains, & Admin Improvements https://www.codexworker.com/post/110/a What developments have taken place today? This day’s work focused on enhancing website analytics tracking and improving the presentation of domain expiration information. Several code refactorings were undertaken to streamline data collection, enhance UI elements, and improve maintainability. Let's break down the changes: 1. **Enhanced Website Analytics:** A significant overhaul was implemented for capturing user activity, including location, visibility changes, cookie management, and session tracking. The introduction of SSE (Server-Sent Events) enables real-time data updates, while cookie checks ensure consistent user identification. Furthermore, a comprehensive set of metrics related to system performance is now collected and reported. 2. **Robust Table Rendering:** A new `Table` class was introduced to encapsulate the logic for generating HTML tables dynamically. This improves code organization and reusability. The domain URL table has been updated to include expiration status indicators using color-coded tags, offering a quick visual assessment of domain health. 3. **Improved Email Generation:** Refactoring of email generation logic within `helper_email_generator` created a more modular design, enhancing testability and future extensibility. A new factory function simplifies instantiation of the EmailGenerator struct. 4. **Admin Route Implementation**: Three new routes were added to the admin section for viewing last login times and accessing administrative logs. 5. **Domain Expiration Display:** Added an API endpoint `/api/domain/expiration` and associated front-end changes to display domain expiration dates in a table format, providing users with critical information about website longevity. This includes parsing and formatting date strings for clear presentation. 6. **Style Adjustments**: Minor style adjustments were made to update background colors for status indicators (`status-ok`, `status-failed`) to improve visual clarity. 7. **Dependency Injection:** The codebase now employs dependency injection, separating concerns and improving modularity through the use of constructor functions to create new service instances. 8. **Code Refactoring**: Several minor code refactorings were performed throughout the project, including updating import statements in a stylesheet and reorganizing controller initialization logic. In summary, today’s changes significantly improved website analytics tracking, enhanced data presentation with clear expiration indicators, streamlined backend processes through dependency injection, and added new administrative functionalities. These improvements collectively contribute to a more robust, informative, and user-friendly experience for visitors and administrators alike. Mon, 07 Apr 2025 23:59:59 +0000 Project Updates: A Day of Refinement and Enhancements https://www.codexworker.com/post/99/a What developments have taken place today? Several significant changes were implemented across various aspects of the project. Firstly, a new newsletter signup form was introduced on the homepage, including basic styling and JavaScript functionality for handling subscriptions. Secondly, a substantial amount of unused CSS – encompassing styles for newsletters, resets, global layouts, flash messages, breadcrumbs, forms, projects, tables, and flip cards – was removed to streamline the codebase. This cleanup improves maintainability and reduces project size. Thirdly, import statements in stylesheets were modified, alongside the initialization of new controllers including `contactController`, `gitlabController`, `homepageController`, `projectController`, `registrationController`, and `rssController`. This restructuring likely enhances component organization and code reuse. Fourthly, project information display was updated with styling for individual listings (light gray background, rounded corners) and revised text labels. Finally, comprehensive website analytics tracking was implemented, utilizing Server-Sent Events (SSE) for real-time data updates, visibility change detection via `navigator.sendBeacon`, enhanced cookie management for unique user identification, detailed browser and session information collection, and data transmission to a designated tracking server. Lastly, the background color of a flip card's back side was adjusted to dark blue with added padding and alignment adjustments removed. These changes are important because they contribute to a cleaner codebase through removal of obsolete styles, improve website functionality with analytics enhancements, increase user engagement by collecting comprehensive data about their browsing experience, and enhance the visual presentation of project information for improved clarity and usability. Mon, 07 Apr 2025 23:59:59 +0000 Website Updates: A Comprehensive Overview https://www.codexworker.com/post/93/a What developments have taken place today? Several updates were made across various aspects of the website’s codebase. A new newsletter signup form was added to the homepage with basic styling and JavaScript integration. Simultaneously, a substantial amount of unused CSS – including reset styles, global layout definitions, flash messages, breadcrumb navigation, contact/project forms, table styles, and flip card styles – was removed to streamline the codebase. A stylesheet update introduced new imports for `columns.scss` and `flip_card.scss`, alongside the addition of several new controllers: `contactController`, `gitlabController`, `homepageController`, `projectController`, `registrationController`, and `rssController`. These changes also adjusted the order of initialized controllers. Furthermore, project information displays were enhanced with styling – including a light gray background and rounded corners – while updating link behavior to open in new tabs with specific source parameters, and changing "More detailed changes" to “Changes in projects”. Significant improvements were made to website analytics tracking. Server-Sent Events (SSE) are now utilized for real-time data updates regarding user location and visit frequency. Visibility change tracking captures session start times and durations when the browser tab becomes invisible. Cookie management ensures unique user identification via random strings if necessary, and general analytics data collection gathers comprehensive information about browser settings, URL parameters, and device type. All collected data is then sent to a designated tracking server via XMLHttpRequest. Finally, system metrics gathering was implemented using Go’s `metrics` package, collecting data on memory usage, garbage collection cycles, goroutine counts, and object allocations/frees. This data is formatted and reported as a structured string for monitoring purposes. These changes collectively contribute to improved user experience through enhanced form functionality, simplified codebase maintenance, more accurate website analytics, and increased system visibility. Mon, 07 Apr 2025 23:59:59 +0000 Automatic information https://www.codexworker.com/post/70/a Hello! Code change information from bot here! During week 2025-03-31 - 2025-04-06 i made 25 commits across 3 of my projects. In total i created 5952 new lines and removed 6998 lines. Mon, 07 Apr 2025 22:01:20 +0000 Domain expiration https://www.codexworker.com/post/71/a Domain expiration list available on remotedaemon.com Mon, 07 Apr 2025 21:02:03 +0000 API Improvements & System Updates: A Recap https://www.codexworker.com/post/109/a What developments have taken place today? Today’s commits focused primarily on improving API responses, error handling consistency, and code organization across several controllers and repositories. Several key improvements were implemented: **1. Enhanced API Responses & Error Handling:** The `InitReponseDetails` function was updated with more descriptive error types and included a `ResponseBodySchema` for better documentation and client-side validation. Login and Device controllers now utilize standardized `ApiError` objects and `RespondWithJSON` consistently, simplifying client handling. New helper functions like `GetBaseDomainAndProtocol` were added to improve URL parsing flexibility. Validator improvements ensured URL input is only the domain name, preventing invalid submissions. **2. Centralized ACL Initialization:** The code now utilizes a centralized method, `acl := acl_rules.InitAcl()`, for initializing the ACL struct across multiple controllers, streamlining management and reducing redundancy. The controllers affected include AdminController, DeviceController, ApiWebsiteController, WebsiteController, and WebsiteURLController. **3. SSL/TLS Certificate Expiration Retrieval:** The `GetExpirationOfDomainCertificate` function was significantly improved by directly querying the server for its certificate using TLS, providing more reliable expiration date retrieval. This includes robust error handling for network issues and a test case utilizing self-signed certificates. **4. Website Repository Enhancements:** The `NewWebsiteRepository` constructor was added to improve code organization. The `LoadWebsiteForUser` method now incorporates better error handling during database queries, logging errors effectively, and handles NULL values gracefully within the result set. **5. Favicon Serving Updates:** The favicon serving mechanism transitioned from `r.Static` to `r.StaticFile`, ensuring proper static file delivery. **6. Dependency Updates:** Several `@esbuild/*` packages and the `gar/promisify` library were updated to their latest versions (0.25.2 and 1.1.3 respectively) for bug fixes and potential performance gains. The `fsevents` package was also updated to version 2.3.3 In essence, these changes collectively contribute to a more robust, maintainable, and reliable API by prioritizing clearer error messages, consistent coding practices, improved data integrity, and streamlined architectural design. The updates enhance the user experience through better form validation and ensure that critical system components like SSL certificate retrieval operate effectively. Sun, 06 Apr 2025 23:59:59 +0000 Infrastructure Updates Ensure Application Stability https://www.codexworker.com/post/108/a What developments have taken place today? The application’s database credentials and email configuration have been updated to accommodate a shift in underlying infrastructure. Specifically, the MySQL username and password for the `remotedaemon_db` database were altered, as was the SMTP server URL and sender address. Email templates and associated migration scripts have been revised to utilize these new settings. Furthermore, the Dockerfile has been updated to employ MariaDB version 11.4.5. These changes ensure seamless operation with the new environment. These updates are vital improvements because they guarantee continued functionality by adapting the application’s connections to its new hosting environment. Updating database credentials and SMTP details prevents connection failures and ensures accurate email delivery. Utilizing a more recent MariaDB version provides enhanced security and stability for the application's data storage. Overall, these modifications are essential for a successful deployment transition. Sat, 05 Apr 2025 23:59:59 +0000 Long day https://www.codexworker.com/post/69/a Long day, but email server is working. Own mail server. Sat, 05 Apr 2025 23:27:09 +0000 Today’s Development Updates: Analytics, GPRC & Security Improvements https://www.codexworker.com/post/107/a What developments have taken place today? This commit improved the accuracy of client IP address capture in analytics data by utilizing a helper function, `helper.GetRealAddr(c.Request)`, instead of directly accessing `c.ClientIP()`. This enhancement provides a more reliable source of client IP information. A new type parameter has been introduced and implemented across several functions, including `BuildUrl`, `BuildStatsUrl`, `BuildUrlTableData`, `prepareAnalyticsData`, `getSearchConfig()`, and `updateChartSelect`. This allows for differentiated analytics data transmission to the gRPC server based on a specified type. The update also includes error handling within the URL construction process and validation of user-selected values from an HTML element. The configuration file has been updated to include a gRPC token, facilitating communication with the gRPC service. Furthermore, changes have been made to the gRPC connection string and authorization header – specifically, shifting the server address and updating the `x-authorization` metadata for enhanced security and connectivity. A short delay is implemented before attempting to establish a gRPC connection. These combined modifications represent significant improvements by enabling flexible analytics data transmission, enhancing data reliability, strengthening security protocols, and streamlining communication with the gRPC service. Fri, 04 Apr 2025 23:59:59 +0000 Today's Analytics Updates: Improved Accuracy & Functionality https://www.codexworker.com/post/98/a What developments have taken place today? Several improvements were implemented regarding analytics data handling and gRPC communication. Firstly, the method for capturing client IP addresses has been enhanced by utilizing a helper function (`helper.GetRealAddr`) instead of directly accessing the web framework's client IP, improving accuracy and reliability. Secondly, changes were made to URL construction functions – `BuildUrl`, `BuildStatsUrl`, `BuildUrlTableData`, `prepareAnalyticsData` and `getSearchConfig()` - by incorporating a type parameter into the generated URLs derived from the `searchObject`. This enables sending different types of analytics data. The `updateChartSelect` function was updated to parse the "type" value from an HTML element, validating its presence. An analytics form has also been added to allow users to select between Javascript tracker and gRPC methods. A configuration file was updated with a new gRPC token. Finally, modifications were made to the gRPC connection string and authorization header, switching to a more specific service name ("analyticsforwebsite_grpc") and updating the authentication metadata for enhanced security. These changes collectively improve data transmission accuracy, expand analytics capabilities by supporting diverse data formats, enhance security through updated credentials, and provide users with greater control over tracking methods. The improvements are vital for ensuring reliable analytics data collection and application functionality. Fri, 04 Apr 2025 23:59:59 +0000 Analytics Enhancements: IP Addresses, gRPC, and User Selection https://www.codexworker.com/post/92/a What developments have taken place today? The analytics data extraction process has been enhanced by implementing a more reliable method for capturing client IP addresses, moving from direct use of the web framework’s `ClientIP()` function to a helper function (`helper.GetRealAddr()`) that retrieves the IP address from the request object. This change improves accuracy and reliability in IP address identification. The code now incorporates a type parameter into URLs constructed for analytics data transmission to the gRPC server. Functions such as `BuildUrl`, `BuildStatsUrl`, `BuildUrlTableData`, `prepareAnalyticsData`, `getSearchConfig()`, and `updateChartSelect` have been updated to include this parameter, derived from the `searchObject`. A validation has also been added for the field. Additionally, a select box was introduced in `analytics_form.html` to facilitate user selection between Javascript tracker and gRPC data transmission. The configuration file has been modified to include a gRPC token. These changes collectively improve the application's analytics capabilities by supporting diverse data formats and tracking methods through the gRPC server, while bolstering security with updated authentication protocols and connection details. Fri, 04 Apr 2025 23:59:59 +0000 Analytics Pipeline Enhancements: IP Capture & gRPC Updates https://www.codexworker.com/post/89/a Today's changes: The analytics data processing pipeline has been enhanced with improved IP address capture and expanded functionality for sending data to the gRPC server. A new helper function, `helper.GetRealAddr()`, is now utilized to retrieve client IP addresses from HTTP requests, ensuring more accurate tracking compared to relying solely on the web framework's client IP information. The application now supports sending different types of analytics data to the gRPC server via a newly implemented type parameter within URL construction. Functions like `BuildUrl`, `BuildStatsUrl`, and `BuildUrlTableData` have been updated to include this parameter, derived from user selections or other configured logic. Related functions such as `prepareAnalyticsData`, `getSearchConfig()`, and `updateChartSelect` were also modified to accommodate the new type parameter along with validation checks. A select box has been added to the analytics form allowing users to choose between Javascript tracker and gRPC data transmission. Furthermore, the gRPC connection string and authorization header have been updated to reflect a change in server address and authentication token for enhanced security and reliability. A short delay is now implemented before establishing the gRPC connection, coupled with an authenticated request. Fri, 04 Apr 2025 23:59:59 +0000 Email server https://www.codexworker.com/post/68/a Thinking about own email server... Self hosted? What could go wrong... Fri, 04 Apr 2025 23:49:42 +0000 New Analytics Tracking System Implemented https://www.codexworker.com/post/106/a What developments have taken place today? A new analytics tracking system has been implemented within the application, designed to capture and transmit website request data for monitoring and analysis. This involves several key changes: 1. **Analytics Middleware:** A Gin middleware function, `TrackAnalyticsForWebsite`, is introduced to collect relevant HTTP request information (cookies, IP addresses, user agents, timestamps, etc.) and format it into an analytics message. This message is then transmitted asynchronously via a gRPC channel to avoid impacting the main application thread's performance. 2. **Asynchronous Analytics Channel:** A dedicated `trackChannel` is established for asynchronous delivery of analytics data. A background process continuously consumes and sends this data to the gRPC server, ensuring responsiveness. 3. **gRPC Infrastructure Setup:** A gRPC client connection and service endpoint are configured, utilizing insecure credentials for development purposes. The endpoint address is set to "analyticsforwebsite\_grpc:15018". 4. **Docker Health Check Enhancement:** The existing Docker health check script has been updated to verify the status of all critical application components (database, Elasticsearch, API, gRPC server) before proceeding with deployment iterations, ensuring system stability. These changes represent a significant improvement by providing a robust and scalable method for gathering detailed request-level analytics data. The asynchronous processing and dedicated gRPC channel ensure that analytics tracking doesn’t introduce performance bottlenecks or block the main application flow. Furthermore, integrating health checks into the Docker process increases operational reliability. This enhanced monitoring capability provides valuable insights for understanding user behavior, identifying potential issues, and optimizing website performance. Thu, 03 Apr 2025 23:59:59 +0000 New Analytics Tracking & System Updates Implemented https://www.codexworker.com/post/97/a What developments have taken place today? This commit implements a new analytics tracking system for the application. It introduces a middleware function, `TrackAnalyticsForWebsite`, that intercepts incoming HTTP requests and gathers relevant data such as cookies, IP addresses, user agents, and request source information. This data is then transmitted asynchronously via a gRPC channel to a dedicated gRPC service (“analyticsforwebsite\_grpc:15018”) for processing or storage. A background process continuously consumes analytics data from the channel and sends it to the gRPC server, preventing blocking of the main application thread. Additionally, the Docker health check script has been updated to verify the health status of all core components (database, Elasticsearch, application, API, and gRPC) before proceeding with deployment or updates, ensuring system stability. The configuration includes the gRPC server address for easy setup. These changes represent an important improvement by introducing a structured and asynchronous approach to collecting user behavior data. The middleware ensures that analytics are captured without impacting performance, while the gRPC channel facilitates efficient communication between the application and a dedicated analytics service. Integrating Docker health checks strengthens system reliability and simplifies deployment processes. Thu, 03 Apr 2025 23:59:59 +0000 Website Analytics System Updates & Improvements https://www.codexworker.com/post/91/a What developments have taken place today? A new system for collecting and transmitting website analytics data has been implemented. This involves several key components: a middleware function (`TrackAnalyticsForWebsite`) that gathers request information (cookies, IP addresses, user agents, etc.) and sends it asynchronously via a gRPC channel to a dedicated service. This asynchronous approach prevents performance bottlenecks in the main application thread. A gRPC client-server setup is configured for communication, utilizing insecure credentials for development purposes. Furthermore, the Docker health check script has been updated to verify the health of all related services (database, Elasticsearch, API, and gRPC) before continuing deployment processes. The middleware utilizes a channel for asynchronous data processing, improving application responsiveness by preventing blocking. The gRPC infrastructure enables efficient and decoupled transmission of analytics data to a centralized location for analysis or storage. Integrating Docker health checks ensures that the system operates reliably by verifying component availability. Secure credentials are included for development purposes, with explicit notes regarding necessary changes in production environments. These improvements are crucial because they establish a robust foundation for comprehensive website analytics tracking. By decoupling data collection from request processing and utilizing asynchronous communication, the application’s performance is maintained while simultaneously providing valuable insights into user behavior and system health. The standardized approach to analytics data transmission facilitates future integration with other tools and platforms. Thu, 03 Apr 2025 23:59:59 +0000 New Analytics Tracking System & Docker Health Checks Implemented https://www.codexworker.com/post/88/a What developments have taken place today? This commit introduces a new analytics tracking system for the application, designed to capture and transmit user interaction data asynchronously. A `TrackAnalyticsForWebsite` middleware is implemented within the Gin framework to gather request-level information such as cookies, IP addresses, user agents, timestamps, event types, device dimensions, website IDs, and authorization tokens. This collected data is then sent via a gRPC channel to a dedicated gRPC server running on port 15018 for processing or storage. To prevent blocking the main request thread, an asynchronous channel (`trackChannel`) has been established and utilized by a background goroutine that continuously processes incoming analytics data and transmits it to the gRPC endpoint. Furthermore, the Docker health check script has been updated to verify the operational status of critical components – database, Elasticsearch, the application itself, the API, and the gRPC server – before proceeding with subsequent steps. Secure credentials are currently set for simplicity but require proper authentication in a production environment. These changes improve data collection efficiency by decoupling analytics processing from request handling, leading to improved responsiveness. The asynchronous nature of the channel prevents performance bottlenecks. Integrating Docker health checks ensures the stability and reliability of the application deployment pipeline by verifying that all necessary components are running correctly before continuing operations. This system provides a robust foundation for gathering valuable insights into user behavior and application performance. Thu, 03 Apr 2025 23:59:59 +0000 gRPC Integration & Containerization Update for Analytics Application https://www.codexworker.com/post/105/a What developments have taken place today? This commit introduces gRPC-based communication for the `analyticsforwebsite` application. It includes the creation of protobuf definitions defining a simple greeting service, alongside server and client implementations utilizing gRPC. A Docker configuration has been established to facilitate consistent deployment of both the gRPC server and client within containerized environments. Furthermore, Git configurations have been added to streamline development workflows inside the containers, including SSH key management and Go module version updates (to 1.22.0) along with a specific protoc compiler version (1.36.6). The changes provide structured data exchange between application components and simplify deployment through containerization. These improvements are significant because they establish a standardized, efficient, and easily manageable method for inter-component communication within the `analyticsforwebsite` application. Utilizing gRPC with Docker containers ensures scalability, maintainability, and consistent operational practices across development and production environments. Tue, 01 Apr 2025 23:59:59 +0000 gRPC Update: New Communication Layer Implemented https://www.codexworker.com/post/96/a What developments have taken place today? This commit introduces a gRPC-based communication layer for the `analyticsforwebsite` application. Key changes include defining protobuf messages and services to facilitate data exchange, implementing both a gRPC server and client for handling greetings, configuring Docker containers to deploy these components, and integrating Git configuration for streamlined development within the container environment. Specifically, a `helloworld.proto` file was created detailing a simple greeting service with associated message types. The server listens on port 15018 and responds to requests with personalized greetings. A client connects to the server, sends a request including a name, and receives the corresponding reply. Docker containers are now defined for both the server and client, incorporating health checks for monitoring and entrypoint scripts for automated startup. Furthermore, Git configuration has been implemented within the containers to simplify code management. Finally, the Go module version was upgraded to 1.22.0, along with a specific protoc compiler version (1.36.6). These changes represent an important advancement by establishing a standardized and efficient method for application components to interact. Utilizing gRPC provides structured data exchange, improves performance through binary serialization, and simplifies deployment via Docker containers, contributing to a more robust and maintainable application architecture. Tue, 01 Apr 2025 23:59:59 +0000 gRPC Update & Containerization for Analytics Application https://www.codexworker.com/post/90/a What developments have taken place today? This commit introduces a gRPC-based communication layer for the `analyticsforwebsite` application. Key improvements include defining protobuf messages and services, implementing both a gRPC server and client, configuring Docker containers to run these components, and establishing Git integration within those containers. Specifically, the project defines a simple greeting service using `helloworld.proto`, creates a server that responds to “SayHello” requests with personalized greetings, and a client that connects to the server and sends a request with a specified name. The deployment is streamlined through Docker containers which include entrypoint scripts for execution, health checks, and access to application files via volumes. Git configuration has been added within the containers to simplify code management. Finally, the Go module version has been updated to 1.22.0, alongside a specific protoc compiler version of 1.36.6. These changes are important because they establish a standardized and efficient method for inter-component communication within the `analyticsforwebsite` application. The use of gRPC provides structured data exchange and facilitates scalability and maintainability by decoupling components and enabling asynchronous communication, while the Docker configuration simplifies deployment and ensures consistent operation across environments. Tue, 01 Apr 2025 23:59:59 +0000 gRPC Infrastructure Update for Analytics Website https://www.codexworker.com/post/87/a What developments have taken place today? This commit introduces a robust gRPC infrastructure for the `analyticsforwebsite` application. New protobuf definitions were created to define a simple greeting service, encompassing message structures for requests and replies. A functional gRPC server implementation was developed, listening on port 15018 and responding to incoming greetings with personalized messages. Complementing this is a corresponding gRPC client implementation that connects to the server, sends requests, and receives responses. Docker configurations have been added to define and manage both the server and client containers, utilizing entrypoint scripts for execution and health checks for monitoring. Furthermore, Git configuration was integrated within the containers to facilitate code management and deployment. Finally, the Go module version has been upgraded to 1.22.0 and a specific protoc compiler version (1.36.6) has been set. These improvements streamline communication between application components through an efficient and standardized gRPC protocol. These changes are important because they establish a modern, scalable, and maintainable architecture for data exchange within the `analyticsforwebsite` application by leveraging gRPC, simplifying deployment with Docker, and incorporating best practices for version control and development environments. Tue, 01 Apr 2025 23:59:59 +0000 Automatic information https://www.codexworker.com/post/66/a Hello! Code change information from bot here! During week 2025-03-24 - 2025-03-30 i made 0 commits across 0 of my projects. In total i created 0 new lines and removed 0 lines. Mon, 31 Mar 2025 21:01:07 +0000 Automatic information https://www.codexworker.com/post/65/a Hello! Code change information from bot here! During week 2025-03-17 - 2025-03-23 i made 32 commits across 2 of my projects. In total i created 3685 new lines and removed 8536 lines. Mon, 24 Mar 2025 22:01:11 +0000 Automatic information https://www.codexworker.com/post/64/a Hello! Code change information from bot here! During week 2025-03-10 - 2025-03-16 i made 57 commits across 10 of my projects. In total i created 10918 new lines and removed 15871 lines. Mon, 17 Mar 2025 20:01:25 +0000 Automatic information https://www.codexworker.com/post/63/a Hello! Code change information from bot here! During week 2025-03-03 - 2025-03-09 i made 65 commits across 11 of my projects. In total i created 12071 new lines and removed 10333 lines. Mon, 10 Mar 2025 22:41:16 +0000 Automatic information https://www.codexworker.com/post/59/a Hello! Code change information from bot here! During week 2025-02-24 - 2025-03-02 i made 91 commits across 9 of my projects. In total i created 16232 new lines and removed 15587 lines. Mon, 03 Mar 2025 22:01:06 +0000 Automatic information https://www.codexworker.com/post/58/a Hello! Code change information from bot here! During week 2025-02-17 - 2025-02-23 i made 76 commits across 8 of my projects. In total i created 13245 new lines and removed 8168 lines. Mon, 24 Feb 2025 22:00:59 +0000 Automatic information https://www.codexworker.com/post/57/a Hello! Code change information from bot here! During week 2025-02-10 - 2025-02-16 i made 100 commits across 7 of my projects. In total i created 14682 new lines and removed 14783 lines. Mon, 17 Feb 2025 21:01:12 +0000 Automatic information https://www.codexworker.com/post/56/a Hello! Code change information from bot here! During week 2025-02-03 - 2025-02-09 i made 76 commits across 2 of my projects. In total i created 7616 new lines and removed 7065 lines. Mon, 10 Feb 2025 21:01:01 +0000 Automatic information https://www.codexworker.com/post/55/a Hello! Code change information from bot here! During week 2025-01-27 - 2025-02-02 i made 76 commits across 7 of my projects. In total i created 9612 new lines and removed 7278 lines. Mon, 03 Feb 2025 23:01:00 +0000 Automatic information https://www.codexworker.com/post/54/a Hello! Code change information from bot here! During week 2025-01-20 - 2025-01-26 i made 47 commits across 6 of my projects. In total i created 9432 new lines and removed 11371 lines. Mon, 27 Jan 2025 20:00:46 +0000 Automatic information https://www.codexworker.com/post/53/a Hello! Code change information from bot here! During week 2025-01-13 - 2025-01-19 i made 46 commits across 5 of my projects. In total i created 7062 new lines and removed 3098 lines. Mon, 20 Jan 2025 22:00:44 +0000 Automatic information https://www.codexworker.com/post/52/a Hello! Code change information from bot here! During week 2025-01-06 - 2025-01-12 i made 1 commits across 1 of my projects. In total i created 9 new lines and removed 1 lines. Mon, 13 Jan 2025 21:00:27 +0000 Automatic information https://www.codexworker.com/post/51/a Hello! Code change information from bot here! During week 2024-12-30 - 2025-01-05 i made 62 commits across 6 of my projects. In total i created 17941 new lines and removed 20850 lines. Mon, 06 Jan 2025 23:00:54 +0000 Automatic information https://www.codexworker.com/post/50/a Hello! Code change information from bot here! During week 2024-12-23 - 2024-12-29 i made 32 commits across 3 of my projects. In total i created 4458 new lines and removed 2964 lines. Mon, 30 Dec 2024 20:12:03 +0000 Automatic information https://www.codexworker.com/post/46/a Hello! Code change information from bot here! During week 2024-12-16 - 2024-12-22 i made 34 commits across 5 of my projects. In total i created 4334 new lines and removed 6498 lines. Mon, 23 Dec 2024 21:00:38 +0000 Automatic information https://www.codexworker.com/post/45/a Hello! Code change information from bot here! During week 2024-12-09 - 2024-12-15 i made 28 commits across 7 of my projects. In total i created 3829 new lines and removed 2196 lines. Mon, 16 Dec 2024 20:00:34 +0000 Automatic information https://www.codexworker.com/post/44/a Hello! Code change information from bot here! During week 2024-12-02 - 2024-12-08 i made 19 commits across 3 of my projects. In total i created 3099 new lines and removed 2658 lines. Mon, 09 Dec 2024 20:00:32 +0000 Automatic information https://www.codexworker.com/post/43/a Hello! Code change information from bot here! During week 2024-11-25 - 2024-12-01 i made 61 commits across 6 of my projects. In total i created 10102 new lines and removed 6378 lines. Mon, 02 Dec 2024 21:00:51 +0000 Automatic information https://www.codexworker.com/post/42/a Hello! Code change information from bot here! During week 2024-11-18 - 2024-11-24 i made 23 commits across 5 of my projects. In total i created 2856 new lines and removed 3441 lines. Mon, 25 Nov 2024 23:00:30 +0000 Automatic information https://www.codexworker.com/post/41/a Hello! Code change information from bot here! During week 2024-11-11 - 2024-11-17 i made 32 commits across 2 of my projects. In total i created 2409 new lines and removed 1415 lines. Mon, 18 Nov 2024 22:00:41 +0000 Automatic information https://www.codexworker.com/post/40/a Hello! Code change information from bot here! During week 2024-11-04 - 2024-11-10 i made 32 commits across 3 of my projects. In total i created 5892 new lines and removed 2420 lines. Mon, 11 Nov 2024 21:00:35 +0000 Automatic information https://www.codexworker.com/post/39/a Hello! Code change information from bot here! During week 2024-10-28 - 2024-11-03 i made 24 commits across 2 of my projects. In total i created 6834 new lines and removed 3815 lines. Sun, 10 Nov 2024 23:18:20 +0000 Automatic information https://www.codexworker.com/post/37/a Hello! Code change information from bot here! During week 2024-10-27 - 2024-11-03 i made 20 commits across 2 of my projects. In total i created 3333 new lines and removed 1684 lines. Sun, 03 Nov 2024 21:00:17 +0000 Automatic information https://www.codexworker.com/post/36/a Hello! Code change information from bot here! During week 2024-10-20 - 2024-10-27 i made 15 commits across 2 of my projects. In total i created 8730 new lines and removed 6020 lines. Sun, 27 Oct 2024 21:00:15 +0000 Automatic information https://www.codexworker.com/post/35/a Hello! Code change information from bot here! During week 2024-10-13 - 2024-10-20 i made 20 commits across 3 of my projects. In total i created 5266 new lines and removed 9079 lines. Sun, 20 Oct 2024 20:00:15 +0000 Automatic information https://www.codexworker.com/post/34/a Hello! Code change information from bot here! During week 2024-10-06 - 2024-10-13 i made 31 commits across 4 of my projects. In total i created 5701 new lines and removed 5297 lines. Sun, 13 Oct 2024 22:32:00 +0000 Automatic information https://www.codexworker.com/post/33/a Hello! Code change information from bot here!. During week 2024-09-29 - 2024-10-06 i made 13 commits across 4 of my projects. In total i created 3686 new lines and removed 1172 lines. Open for new projects! Sun, 06 Oct 2024 23:15:20 +0000 Automatic posts https://www.codexworker.com/post/32/a I need to create automatic posts, which will create weekly summary of my commits across all my personal projects Sun, 06 Oct 2024 21:34:48 +0000 basic android app released https://www.codexworker.com/post/31/a basic android app released, not in play store yet, maybe never will be... Account needs to be register on web, then used on android. Beta testers welcomed. Mon, 09 Sep 2024 20:07:12 +0000 updating push notifications https://www.codexworker.com/post/30/a android app on remotedaemon in progress. useful alternative to web. Fri, 06 Sep 2024 22:21:09 +0000 remotedaemon errors https://www.codexworker.com/post/28/a show error on homepage, if user is logged, so users see in realtime if one of their websites is down / up and what is the problem there Mon, 19 Aug 2024 21:15:56 +0000 email verification https://www.codexworker.com/post/27/a email verification added, so now users can see only verified users (at least by email). Also side project verification enabled. Sat, 17 Aug 2024 21:56:06 +0000 colors to graph https://www.codexworker.com/post/26/a new colors to website list red - your website is down whole day orange - at least once per day was down green - website is up and running whole day Wed, 14 Aug 2024 21:10:32 +0000 colors to synopsee https://www.codexworker.com/post/25/a synopsee now have colors representing increase or decrease or watched statistics Mon, 12 Aug 2024 20:24:20 +0000 partnerupzone https://www.codexworker.com/post/24/a profile improved, location added .. tiny improvements over past few days Sun, 11 Aug 2024 20:13:31 +0000 partnerupzone.com https://www.codexworker.com/post/23/a i released new version of partnerupzone, you can login/register, create your profile, offer services and present your side projects. it is still in alpha, but this is core, which should work. also there is a realtime chat, which should also work, so people can contact other people. Tue, 06 Aug 2024 21:39:22 +0000 alpha version https://www.codexworker.com/post/22/a just quick idea, header checker + later security checker. Sat, 03 Aug 2024 22:22:44 +0000 improvements https://www.codexworker.com/post/21/a little bit of improvements across all projects, most of them are now at 100% speed, based on https://pagespeed.web.dev/ Tue, 30 Jul 2024 23:18:29 +0000 work on side projects https://www.codexworker.com/post/20/a during nights and weekends i still work on side projects. More specific updates will come maybe in few month. In meantime, you can checkout other projects and send me list of improvements, which you could consider important. Fri, 12 Jul 2024 20:28:55 +0000 rss https://www.codexworker.com/post/19/a rss on https://www.codexworker.com/rss Mon, 24 Jun 2024 22:34:20 +0000